Friday, December 12th, 2014

Building a Gopher client in Django

Have you ever wanted to enable your users to access your Gopher server without the need of them installing a browser plugin, installing a standalone software, or using a third part proxy service? Now you can, by adding this simple Django app to any existing Django project. It doesn't depend on anything by my updated gopherlib.py, and the following app which I will explain here. I have already developed it and deployed it, so I will not be covering every single detail. For the most part, it's a very basic Django app that only uses function-based views to get the job done. So, this app might be good as a starting point for people new to Django, as there's nothing terribly complex going on here. You can see a fully working example here. First let's cover the urls.py:

from django.conf.urls import patterns, url

urlpatterns = patterns('djgopher.views',
    url(r'^$', 'root_menu', name='gopher_root_menu'),
    url(r'^RemoteDoc$', 'remote_document', name='gopher_remote'),
    url(r'^(?P<selector>.+)$', 'selector', name='gopher_selector'),
)

This isn't the most elegant way to grab the selector from the URL, but it gets the job done. This may have some security implications behind it, I am hoping that my code which handles this selector can't be easily exploited. Feel free to point out to me any possible security concerns and a way to fix it. Let's move onto the views.py:

from djgopher.gopherlib import Gopher, GopherMenu, TextFile
from django.shortcuts import render
from django.http import HttpResponse, Http404, HttpResponseRedirect
from PIL import Image
from StringIO import StringIO

def root_menu(req):
    g = Gopher('gopher.veroneau.net')
    menu = g.get_root_menu()
    return render(req, 'djgopher/menu.html', {'menu':menu.get_data(), 'title':'Root Menu'})

Here is the first function based view, it's only task is to display the initial gopher menu to the user. I did hardcode my own Gopher server here, but for most use cases, you can actually put in 127.0.0.1 to refer to the local Gopher server. Here, we initialize the Gopher class, and obtain the root menu of that server. Then we send it off to a template which we will get into shortly.

def selector(req, selector):
    if req.META['QUERY_STRING'] != '':
        selector += '?'+req.META['QUERY_STRING']
    gtype = selector[0]
    query = req.POST.get('query', None)
    if gtype == '7' and query is None:
        return render(req, 'djgopher/search.html', {'selector': selector, 'title':selector[1:]})
    g = Gopher('gopher.veroneau.net')
    data = g.get_selector(selector, query)
    if isinstance(data, GopherMenu):
        return render(req, 'djgopher/menu.html', {'menu':data.get_data(), 'title':selector[1:]})
    elif isinstance(data, TextFile):
        return HttpResponse('%s' % data, mimetype='text/plain')
    elif gtype == 'I':
        typ = selector[-3:].lower()
        if typ not in ('png', 'jpg', 'jpeg'):
            typ = Image.open(StringIO(data)).format.lower()
        return HttpResponse(data, mimetype='image/%s' % typ)
    elif gtype == 'g':
        return HttpResponse(data, mimetype='image/gif')
    elif gtype == '9':
        return HttpResponse(data, mimetype='application/octet-stream')
    else:
        return HttpResponse('Unhandled.', mimetype='text/plain')

This function based view handles the actual rendering of the chosen selector. Here we check for a query string and append it to the selector so that the Gopher server can manage it. Then we check if the item type is a search, if so, then we display a special template if no query came in. This template is a basic form that will prompt the user for their search query. Finally, if we're still processing here, we grab our selector from the server and check what type we are handling. Technically, we should check if we support the type before even downloading any data to save bandwidth, but for now this should be fine. I believe I will update it shortly so that the item type is properly checked beforehand. Depending on the item type, we either display a menu, a text file, image, or perform a binary download. We also use the PIL library if the image file doesn't have a valid extension to make sure we serve the correct information to the browser.

def remote_document(req):
    url = req.GET.get('uri', None)
    if url is None:
        raise Http404
    hostname, path = url.split('/',1)
    host, port = hostname.split(':')
    if path[0] != '0':
        return HttpResponseRedirect('http://gopher.floodgap.com/gopher/gw?gopher://%s:%s/%s' % (host, port, selector))
    g = Gopher(host, int(port))
    data = g.get_selector(path)
    if isinstance(data, TextFile):
        return HttpResponse('%s' % data, mimetype='text/plain')
    else:
        return HttpResponseRedirect('http://gopher.floodgap.com/gopher/gw?gopher://%s:%s/%s' % (host, port, selector))

This final function based view is to enable users to retrieve remote documents, such as this one here. I only want to enable text document linking, not anything else to limit bandwidth and overall complexity. There are other clients and proxies out there that can easily grab resources from any Gopher server, so it's best to leverage those services for this.

The Templates

The templates are pretty dead simple, as this is a very basic client application, nothing too fancy is required to display Gopher content. Here is our base.html template:

<html>
  <head><title>{{title}} | Django Gopher Client</title></head>
  <body>
    {{title}} | Django Gopher Client<hr/>
    {% block content %}{% endblock %}
  </body>
</html>

See? Nothing too complex, not even a stylesheet, although feel free to skin your own client all you want. Here is the template which renders a Gopher menu for us in HTML:

{% extends "djgopher/base.html" %}
{% load gopher %}

{% block content %}
<pre>{% for item in menu %}{% if item.type == 'i' or item.type == '3' %}{{item.type|icon:STATIC_URL}}{{item.name}}
{% else %}<a href="{{item|selector}}">{{item.type|icon:STATIC_URL}}{{item.name}}</a>
{% endif %}{% endfor %}</pre>
{% endblock %}

We are using pre tags here to ensure the content in the menu is rendered correctly. Lots of Gopherholes, including my own use ASCII art in the menus. Gopherpedia is another example of this. This template uses some custom template filters to enable us to dynamically place in URLs and icons using Python code. Like they say, the template is not for logic! Our final template is the search.html template:

{% extends "djgopher/base.html" %}

{% block content %}
<form action="{% url gopher_selector selector %}" method="post">{% csrf_token %}
Query: <input type="text" name="query"/><input type="submit" value="Go"/>
</form>
{% endblock %}

The search template is pretty normal, just a basic HTML post form. Now, finally here is the template tag library to render those filters:

from django import template
from django.core.urlresolvers import reverse
from django.utils.safestring import mark_safe

register = template.Library()

ICON_TYPES = {
    '0': 'text.png',
    '1': 'dir.png',
    'h': 'html.png',
    '9': 'binary.png',
    'I': 'image.png',
    'g': 'image.png',
    '7': 'search.png',
    '3': 'error.png',
    '8': 'telnet.png',
}

@register.filter
def selector(value):
    if value['selector'][:5] == 'hURL:':
        return value['selector'][5:]
    if value['host'] != 'gopher.veroneau.net':
        uri = '%s:%s/%s' % (value['host'], value['port'], value['selector'])
        if value['type'] == '0':
            return '%s?uri=%s' % (reverse('gopher_remote'), uri)
        elif value['type'] == '8':
            return 'telnet://%s:%s/' % (value['host'], value['port'])
        else:
            return 'http://gopher.floodgap.com/gopher/gw?gopher://%s' % uri
    return reverse('gopher_selector', args=[value['selector']])

@register.filter
def icon(value, static_url):
    try:
        return mark_safe('<img src="%sdjgopher/%s"/>' % (static_url, ICON_TYPES[value]))
    except KeyError:
        return ''

The first dictionary you see there is to make selecting the icon like a case-select, which is much more efficient than a ton of if statements. The first filter here, selector is used to resolve a menu item into a URL that WWW browsers can understand. Here, we check various attributes based on the Gopher standard and write an HTTP compatible URL link the user can click on. Anything from the supported Gopher server is handled internally by our selector view. Anything that has an external host is handled according to it's type. This makes it so that all Gopher menu items work regardless if the local client supports remote servers. Finally, we have the simple icon filter that does nothing more by write out a proper IMG tag with the icon for the menu item.

And there you have it, a fully functional Gopher client all done from within a Django app! So now you have no excuse not to run your own Gopher server. For some interesting places to visit on Gopher, check out my Bookmarks.

As mentioned in my previous post about Gopher, I will soon be making all my Python Diary content available in Gopherspace, it will contain both full HTML versions and text versions. So, if you ever wanted to try browsing a less distracting side of the Internet where advertisers and trackers can't find you, now is a great time! Over time, I am planning on developing many gateway applications for the Internets most popular online locations, such as Twitter, and various news outlets. I think I'd personally like to add RSS feeds to my Gopherspace to enable the easy reading of many popular Python blogs from Planet Python. If your on Android, be sure to check out the updated Gopher client, which can really save you on your mobile bandwidth!

About Me

My Photo
Names Kevin, hugely into UNIX technologies, not just Linux. I've dabbled with the demons, played with the Sun, and now with the Penguins.




Kevin Veroneau Consulting Services
Do you require the services of a Django contractor? Do you need both a website and hosting services? Perhaps I can help.

This Month

If you like what you read, please consider donating to help with hosting costs, and to fund future books to review.

Python Powered | © 2012-2014 Kevin Veroneau