My Org-Mode Server
I went ahead and started working out how to get my todo list online. I started off pretty simple and ended up with a relatively nice system. The basic idea is that I can push my org files to my webserver and edit them. Likewise, I can pull from the server. It started with some simple paver scripts that uploaded the files and quickly became an actual application.
Here is the paver file for some of the operations:import os from mercurial import commands, ui, hg from paver.easy import * import subprocess IONROCK_HG = 'ssh://eric@ionrock.org/path/to/todos/' REMOTE_TODO = IONROCK_HG # '/local/dev/path/to/todos' @task def server(): import cherrypy cherrypy.tree.graft(TodoServer(base_url='/'), '/') cherrypy.quickstart() @task def create_repo(): cmd = subprocess.call("fab create_repo:hosts='ionrock.org'", shell=True) @task def commit(): conf = ui.ui() user = conf.username() repo = hg.repository(conf, '.') files = [f for f in os.listdir('.') if f.endswith('.org')] commands.add(conf, repo, *files) commands.commit(conf, repo, addremove=True, message='Syncing org files') commands.push(conf, repo, REMOTE_TODO) @task @needs('commit') def pull(): conf = ui.ui() user = conf.username() repo = hg.repository(conf, '.') commands.pull(conf, repo, REMOTE_TODO) commands.update(conf, repo) @task @needs('commit') def update(): cmd = subprocess.call("fab update_todos:hosts='ionrock.org'", shell=True)The server task was for starting the eventual web application for development. The commit task just automatically commits the current org files and pushes them to the remote server. The pull command does the commit first, then pulls from the remote server. These two commands uses the mercurial libraries to work with the mercurial repos. The create_repo was just a simple task to create an mercurial repo. More interesting is the update task which updates the remote todo mercurial repo. I'm using fabric for this aspect. It was all really easy. Here is the fabfile:
from fabric import run
def update_todos():
run('cd /home/eric/htdocs/todo && hg up')
def create_repo():
run('cd /home/eric/htdocs/todo && hg init')
Hopefully it is really clear what is happening here. Fabric lets you run commands via ssh on a remote server. The actual todo server is a bit longer but also pretty simple.
import os
import re
import posixpath as path
import difflib
from selector import Selector
from webob import Response, Request
from webob.exc import *
from mercurial import commands, ui, hg
import datetime
class TodoFile(object):
def __init__(self, fn):
self.fn = fn
self.html_diff = difflib.HtmlDiff()
self.diff = difflib.Differ()
self.matcher = difflib.SequenceMatcher()
lines = [l for l in open(fn, 'r')]
self.matcher.set_seq2(lines)
def _hg(self):
conf = ui.ui()
user = conf.username()
repo = hg.repository(os.path.dirname(self.fn))
return conf, repo, user
def __str__(self):
return ''.join(self.read())
def read(self):
return [l for l in open(self.fn, 'r')]
def write(self, new):
f = open(self.fn, 'w')
clean = re.sub('\r', '', new)
f.write(new)
f.close()
conf, repo, user = self._hg()
date = datetime.datetime.now().strftime('%m-%d-%y %H:%M')
commands.commit(conf, repo, message='Web write on %s' % date)
def is_different(self, new):
self.matcher.set_seqs(new.split('\n'), self.read())
def diff_txt(self, new):
return list(difflib.context_diff(new.split('\n'), self.read()))
def diff_html(self, new):
return self.html_diff.make_file(self.read(), new.split('\n'))
class TodoStore(object):
def __init__(self, directory):
self.dir = os.path.abspath(directory)
def get_todo(self, name):
for fn in os.listdir(self.dir):
if fn.endswith('.org') and (fn[:-4] == name):
return TodoFile(os.path.join(self.dir, fn))
return false
def all(self):
return [fn[:-4] for fn in os.listdir(self.dir) if fn.endswith('.org')]
class Auth(object):
def __init__(self, creds, login_url, success_url=None):
self.login_url = login_url
self.success_url = success_url
self.creds = creds
def __call__(self, f):
def func(env, sr):
sess = env['beaker.session']
if sess.get('auth.user'):
return f(env, sr)
req = Request(env)
sess['auth.after_login_url'] = req.url
sess.save()
return HTTPSeeOther(location=self.login_url)(env, sr)
return func
def login(self, env, sr):
res = Response()
sess = env['beaker.session']
flash = sess.get('flash', '')
if flash:
sess['flash'] = ''
sess.save()
res.write('''<div>%s</div>
<form action="%s" method="post">
<label for="username">Username</label>
<input type="text" name="username" value=""><br />
<label for="password">Password</label>
<input type="password" name="password" value=""><br />
<input type="submit" value="login" />
</form>''' % (flash, self.login_url))
return res(env, sr)
def handle_login(self, env, sr):
req = Request(env)
post = req.POST
sess = env['beaker.session']
if post.get('username') and post.get('password'):
if self.creds.get(post['username']):
if self.creds[post['username']] == post['password']:
sess['auth.user'] = post['username']
url = sess.get('auth.after_login_url', self.success_url)
sess.save()
return HTTPSeeOther(location=url)(env, sr)
sess['flash'] = 'Error logging in.'
sess.save()
return HTTPSeeOther(location=self.login_url)(env, sr)
class TodoServer(object):
def __init__(self, **config):
self.conf = {
'todo_dir': os.path.dirname(os.path.abspath(__file__)),
}
self.conf.update(config or {})
self.auth = Auth(self.conf.get('creds', {}),
self.url('login'),
self.url())
self.store = TodoStore(self.conf['todo_dir'])
self.router = Selector([
('[/]', {'GET': self.listing}),
('/login[/]', {
'GET': self.auth.login,
'POST': self.auth.handle_login
}),
('/{name}/edit[/]', {
'GET': self.edit,
'POST': self.auth(self.update)
}),
('/{name}[/]', {'GET': self.read}),
])
def url(self, extras=None):
extras = extras or ''
if isinstance(extras, list):
extras = '/'.join(extras)
return path.join(self.conf['base_url'], extras)
def _header(self):
return '''<html><head>
<title>org todo server</title>
<style type="text/css">
body {
font-size: 2em; font-family: sans-serif;
}
</style>
'''
def _footer(self):
return '''</body></html>'''
def edit(self, env, sr):
res = Response()
req = Request(env)
name = req.urlvars['name']
td = self.store.get_todo(name)
res.write(self._header())
res.write('''
<form action="%s" method="post">
<input type="submit" name="submit" value="save" /><br />
<textarea rows="50" cols="80" name="new_body">%s</textarea>
</form>
''' % (self.url('%s/edit' % name), str(td)))
res.write(self._footer())
return res(env, sr)
def update(self, env, sr):
req = Request(env)
name = req.urlvars['name']
new_body = req.POST['new_body']
todo = self.store.get_todo(name)
todo.write(new_body.strip())
location = self.url('%s' % name)
return HTTPSeeOther(location=location)(env, sr)
def read(self, env, sr):
res = Response()
req = Request(env)
name = req.urlvars['name']
res.write(self._header())
res.write('''
Home | Edit
<hr />
<pre>''' % (self.url(), self.url('%s/edit' % name)))
td = self.store.get_todo(name)
res.write(str(td))
res.write('</pre>')
res.write(self._footer())
return res(env, sr)
def listing(self, env, sr):
res = Response()
res.write(self._header())
res.write('<ul>\n')
for f in self.store.all():
res.write('<li>%s</li>\n' % (self.url(f), f))
res.write('</ul>\n')
res.write(self._footer())
return res(env, sr)
def __call__(self, env, sr):
return self.router(env, sr)
This is a WSGI app simply because I'm using WSGI for my main application. I save a bit of memory by running all my smaller apps via one WSGI server (CherryPy), which makes a difference as I use a VPS. One observation I made is that things would have been simpler had I been able to use CherryPy. Things like sessions, form processing and even URL routing would have been built in and made the whole thing a lot simpler in terms of dependencies and actual code. This also made me realize what the problem is building applications with WSGI. You really need a framework. I don't mean Pylons, web.py or some other WSGI framework. But you will undoubtedly write some glue code to help handle things like request and response objects that help to deal with form handling, sessions and cookies. It is nice to know that it is so easy to create these micro frameworks, but at the same time, it is clear that people would be making bad decisions. I only say that because I'm one of them. When I think of the micro frameworks I've written throughout the past few years, it is clear that I've had to experiment quite a bit. WebOb was a helpful library for sure, but the API you build translating the request to a WebOb Request means breaking WSGI at some level. That means that you've lost the advantages of WSGI as an API for your application. In my mind, it makes me wonder why then the app was written with WSGI in the first place as there is a solid and proven API already built with something like CherryPy. I doubt I'll rewrite my whole site anytime soon, but if I do, the application framework will most definitely revolve around the framework rather than WSGI. The advantages that I believed were present ended up being much less than I thought. Having a tool like CherryPy manages to take care of the generic aspects enough while letting me use more opinionated aspects such as templating or databases. You could most certainly substitute your framework of choice, but for me CherryPy is making more and more sense.
