February 1st, 2014

Build a website in ncurses

Tutorial for: urwid

Ever wanted to make a truly geeky website using nothing more than ncurses terminal technology? However, you still wanted it to be viewable by the masses whom use Firefox, Chrome, Safari, and IE? This tutorial will get you started on that journey!

This tutorial is not for the novice, and does require shell access and the ability to run daemons and fork processes. This is more suited to someone who owns their own server, rents one, or has a VPS with root access online.

The first thing you need is a local development environment of sorts to make sure the colors and everything else functions in a browser-based VT100 terminal emulator. For this, you will need a Linux workstation, as Shellinabox is currently unavailable on the Windows platform. So so ahead and either install the binary or compile the source for Shellinabox. You can test the app locally via your terminal if you don't want to install shellinabox locally, however testing through Shellinabox is highly recommended!

At this point, you should test out shellinabox to confirm that it is functioning, read their documentation for how to do this. Once shellinabox works, you now need to create a container page to place the shell into, this is where you can place a graphical logo, or some other HTML content like a link to the telnet interface. Here is an example from my profile site to get you started:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8" />
    <title>IAmKevin.CA</title>
    <meta name="keywords" content="kevin veroneau,kevin,veroneau,profile,iamkevin,canada,python,django" />
    <meta name="description" content="Kevin Veroneau's online profile website." />
    <meta name="author" content="Kevin Veroneau" />
    <link href='http://fonts.googleapis.com/css?family=Monofett|Jura|Eater+Caps' rel='stylesheet' type='text/css'>
    <link rel="stylesheet" href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.16/themes/dot-luv/jquery-ui.css" type="text/css" media="all" />
    <link rel="stylesheet" href="myprofile.css" type="text/css" media="all" />
  </head>
  <body>
  <div class="title">IAMKEVIN.CA</div>
  <div class="tagline">I own my personal information</div>
  <iframe src="http://localhost:4200/" style="width: 100%; height: 450px;" seamless="seamless" frameborder="0"></iframe>
  <center>
    Using a text-based browser and not seeing anything?  View the content via <a href="telnet://iamkevin.ca:5199">telnet://iamkevin.ca:5199</a>
    <address><a href="http://www.python.org/">Python</a> Powered | &copy; 2011-2014 Kevin Veroneau</address>
  </center>
  </body>
</html>

Feel free to adjust the iframe's dimensions to your liking. Here is a basic configuration command to launch shellinabox:

$ shellinaboxd -t -s /:kveroneau:kveroneau:HOME:app1

The program to launch there is app1, which has to be in the path due to a currently known bug with shellinabox. Here is an example /usr/local/bin/app1:

#!/usr/bin/python

import sys, os

APP_DIR = '/home/kveroneau/PythonApps/app1'
sys.path.append(APP_DIR)
os.chdir(APP_DIR)

import main

This sets up the required environment for your application, you will need to tweak this to your own settings.

With all that configuration out of the way, now it's time to dive into building the actual app. To make using ncurses and terminal controls much easier, you should use the urwid package. However using urwid is not required, you can build a CLI or use another ncurses library, or use ncurses directly!

Okay, so this is what all of you have been waiting for, the actual source code for my profile website, here it is:

import urwid
import os
import sys
import datetime
from email.mime.text import MIMEText
import smtplib

def sendmsg(to, subject, body):
    """
    This function is used to send an email out, it is used for the contact form.
    """
    msg = MIMEText(body)
    msg['Subject'] = subject
    msg['From'] = '*****@****'
    msg['To'] = to
    s = smtplib.SMTP('localhost')
    s.sendmail(msg['From'], [to], msg.as_string())
    s.quit()

class PythonLogo(urwid.Widget):
    _sizing = frozenset([urwid.FIXED])

    def __init__(self):
        """
        Create canvas containing an ASCII version of the Python
        Logo and store it.
        """
        blu = urwid.AttrSpec('dark blue', 'default')
        yel = urwid.AttrSpec('brown', 'default')
        width = 17
        self._canvas = urwid.Text([
            (blu, "     ______\n"),
            (blu, "   _|_o__  |"), (yel, "__\n"),
            (blu, "  |   _____|"), (yel, "  |\n"),
            (blu, "  |__|  "), (yel, "______|\n"),
            (yel, "     |____o_|")]).render((width,))

    def pack(self, size=None, focus=False):
        """
        Return the size from our pre-rendered canvas.
        """
        return self._canvas.cols(), self._canvas.rows()

    def render(self, size, focus=False):
        """
        Return the pre-rendered canvas.
        """
        urwid.fixed_size(size)
        return self._canvas

class BigText:
    """
    Display the website logo with a "fake" loading to ensure nostalgia.
    """
    palette = [
        ('logo', 'dark green', 'black'),
        ('default','light gray','black'),
    ]
    def main(self):
        """
        This function here is the main entry point into this part of the app.
        It creates our BigText, and places it on the ncurses canvas.
        """
        bt = urwid.BigText(('logo', 'IAmKevin.CA'), urwid.Thin6x6Font())
        bt = urwid.Padding(bt, 'center', None)
        self.text = urwid.Text('', align='center')
        p = urwid.Pile([bt, self.text])
        p = urwid.Filler(p, 'middle', None, 7)
        loop = urwid.MainLoop(p, self.palette, unhandled_input=self.unhandled)
        loop.set_alarm_in(1, self.trigger, 0)
        loop.run()
    def trigger(self, main_loop, d):
        """
        This is a trigger to both display the "fake" loading, and then to exit this class.
        """
        if d == 0:
            self.text.set_text(('default', 'Loading, please wait...'))
            main_loop.set_alarm_in(2, self.trigger, 1)
        elif d == 1:
            raise urwid.ExitMainLoop()
    def unhandled(self, key):
        """
        This is the function that is run for every key press which a widget doesn't handle.
        Currently, this lets yours truly bypass the annoying fake loading...
        """
        if key == 'f8':
            raise urwid.ExitMainLoop()

class PythonPowered:
    """
    This class displays a nice ASCII Python Powered on the ncurses display.
    """
    def main(self):
        logo = PythonLogo()
        logo = urwid.Padding(logo, 'center', None)
        p = urwid.Pile([logo, urwid.Text(('default','Python Powered'), align='center')])
        p = urwid.Filler(p, 'middle', None, 5)
        loop = urwid.MainLoop(p, [('default','light gray','black')], unhandled_input=self.unhandled)
        loop.set_alarm_in(1, self.trigger)
        loop.run()
    def trigger(self, main_loop, d):
        """
        This is our timed trigger to exit this class.
        """
        raise urwid.ExitMainLoop()
    def unhandled(self, key):
        """
        This is the function that is run for every key press which a widget doesn't handle.
        Currently, this lets yours truly bypass the annoying fake loading...
        """
        if key == 'f8':
            raise urwid.ExitMainLoop()

class MainScreen:
    """
    This is the main application class, what the end-user will be navigating, and what will be
    in memory.  I kept this seperate from the splash classes to converve memory, as the previous classes
    by the time this one loads would have been nicely GC'd.
    """
    palette = [
        ('body','light gray','dark blue', 'standout'),
        ('reverse','dark red','light gray'),
        ('header','light gray','dark red', 'bold'),
        ('footer','light gray','dark red', 'bold'),
        ('tab','light gray','dark red'),
        ('hacker','dark green', 'black'),
        ('editfc','light gray', 'dark blue', 'bold'),
        ('editbx','light gray', 'dark blue'),
        ('editcp','black','light gray', 'standout'),
        ('form','black', 'light gray'),
        ('buttn','black','dark cyan'),
        ('buttnf','light gray','dark blue','bold'),
    ]
    tab_names = ['Home', 'Wall', 'Contact', 'Interests', 'About']
    status_tpl = 'IAmKevin.CA v0.1 |'
    app_state = 'NORMAL'
    def main(self):
        """
        This is the main entry point of the application, it sets up the initial frame and the mainloop.
        """
        self.current_tab = self.tab_names[0]
        text_footer = '%s Click the navigation tabs on the top. | F1: Help' % self.status_tpl
        self.status = urwid.Text(text_footer)
        footer = urwid.AttrWrap(self.status, 'footer')
        self.frame = urwid.Frame(self.page_home(), header=self.tabs(), footer=footer)
        self.loop = urwid.MainLoop(self.frame, self.palette, unhandled_input=self.unhandled)
        self.loop.run()
    def tabs(self):
        """
        This function is responsible for rendering the upper tabs, and controls which tab_names
        is currently highlighted.  There are better ways to do this with urwid, but I found this
        to work well for my purposes.
        """
        self.status.set_text('%s Click the navigation tabs on the top. | F1: Help' % self.status_tpl)
        btn_list = []
        for tab in self.tab_names:
            btn = urwid.Button(tab, self.button_press)
            if tab == self.current_tab:
                btn = urwid.AttrWrap(btn, 'body', 'reverse')
            else:
                btn = urwid.AttrWrap(btn, 'tab', 'reverse')
            btn_list.append(btn)
        return urwid.AttrWrap(urwid.GridFlow(btn_list, 13, 3, 0, 'left'), 'header')
    def put_grid(self, data, size=20):
        """
        This function takes in list of data to be formatted on the screen, as seen in
        both the "Interests" and "Home" pages.
        """
        widgets = []
        for info in data:
            widgets.append(urwid.Text(info))
        return urwid.Padding(urwid.GridFlow(widgets, size, 0, 0, 'left'), 'left', size*2)
    def page_home(self):
        """
        This is the display view for the "Home" page, it is only in memory for as long as the
        user is on the page, otherwise it is GC'd to prevent the process memory from becoming
        to large.  In a traditional local app, it is best to keep these views in memory for
        efficiency.
        """
        my_info = [
            'Name:','Kevin',
            'Sex:','Guy',
            'Age:','30',
            'Birthday:','Month of May',
            'Eye Colour:','Blue',
            'Hair Colour:','Black',
            'Modified:','Jan 31st, 2014'
        ]
        bt = urwid.Padding(urwid.BigText('IAmKevin.CA', urwid.Thin6x6Font()), 'left', None)
        l = [bt, self.put_grid(my_info)]
        return urwid.AttrWrap(urwid.ListBox(urwid.SimpleListWalker(l)), 'body')
    def put_text(self, text_list):
        """
        This takes a list of strings and renders them into a compatible urwid widget.
        """
        l = []
        for txt in text_list:
            l.append(urwid.Text(txt))
        return urwid.ListBox(urwid.SimpleListWalker(l))
    def page_wall(self):
        """
        Coming soon!  This will be more like a GuestBook of sorts, where there will be a basic
        form on top, and entering any text will have it appended to a text file.
        The text file will then be rendered using the put_text function.
        """
        return urwid.AttrWrap(self.put_text(['Coming soon...']), 'body')
    def page_contact(self):
        """
        This is the WORKING contact form, it takes input from the end-user and emails the data
        provided to any email address given in the code here.
        """
        self.email = urwid.Edit(('editcp','Your Email:'))
        self.subject = urwid.Edit(('editcp','Subject:'))
        self.message = urwid.Edit(('editcp','Message:'), multiline=True)
        self.msg = urwid.Text('')
        blank = urwid.Divider()
        l = [
             blank,blank,urwid.Text('Contact me'),urwid.Divider('='),
             urwid.Padding(self.msg, ('fixed left',10), 50), blank,
             urwid.Padding(urwid.AttrWrap(self.email, 'editbx', 'editfc'), ('fixed left',10), 50),
             blank,
             urwid.Padding(urwid.AttrWrap(self.subject, 'editbx', 'editfc'), ('fixed left',10), 50),
             blank,
             urwid.Padding(urwid.AttrWrap(self.message, 'editbx', 'editfc'), ('fixed left',10), 50),
             blank,blank,
             urwid.Padding(urwid.AttrWrap(urwid.Button('Send Message', self.send_message), 'buttn', 'buttnf'), ('fixed left',10), 20),
        ]
        listbox = urwid.ListBox(urwid.SimpleListWalker(l))
        self.status.set_text('%s UP / DOWN / Mouse to navigate fields.' % self.status_tpl)
        return urwid.AttrWrap(listbox, 'form')
    def send_message(self, button):
        """
        This function is called when the user presses the "Send Message" button from the above view.
        It confirms that all fields were filled out, however it doesn't confirm the fields are valid.
        It is very simple to use a RegEx to validate the email field, but since I know this form
        will be difficult to spam, I didn't both putting in the check ;)
        """
        if self.email.edit_text and self.subject.edit_text and self.message.edit_text:
            self.msg.set_text('')
            body = 'From: %s\n\n%s' % (self.email.edit_text, self.message.edit_text)
            sendmsg('*****', self.subject.edit_text, body)
            txt = urwid.BigText(('reverse', 'Message Sent!'), urwid.Thin6x6Font())
            popup = urwid.Overlay(txt, self.frame, 'center', None, 'middle', None)
            self.loop.widget = popup
            self.loop.set_alarm_in(1, self.trigger)
        else:
            self.msg.set_text(('header', 'Please be sure to fill out all fields.'))
    def page_interests(self):
        """
        This view is for the "Interests" page, and is similar to the "Home" page.
        """
        my_interests = [
            'Desktop Environment:','K Desktop Environment, IceWM, WindowMaker, UNIX Desktop Environment',
            'Preferred Web Framework:','Django',
            'Places:','Toronto',
            'Food:','Sushi, Seafood, Rice Milk, Peaches',
            'Favorite Language:','Python',
            'Python IDE:','Kate, Nano, Eclipse',
            'Latest Gadget:','Wii U Zelda Bundle',
            'Movies','The Matrix, Death Becomes her, The NET',
            'Latest Game','A Link Between Worlds',
            'Favorite Games','Ultima Series, Zelda Series, Mario 3D World',
            'OS of choice','Debian GNU/Linux',
            'Outdoors','Traveling, Walking long distances, hiking in the countryside and woods',
        ]
        l = [self.put_grid(my_interests, 40)]
        return urwid.AttrWrap(urwid.ListBox(urwid.SimpleListWalker(l)), 'hacker')
    def page_about(self):
        """
        This is the "About" page, where it just reads in a text file and displays it in a scrollable view.
        """
        doc = [x.replace('\n','') for x in open('docs/about.txt', 'r').readlines()]
        self.status.set_text('%s UP / DOWN / PAGE UP / PAGE DOWN scroll.' % self.status_tpl)
        return urwid.AttrWrap(self.put_text(doc), 'body')
    def trigger(self, main_loop, d):
        """
        This is called after we place the "Message Sent" box onto the screen, it clears the fields and removed the box.
        """
        self.email.edit_text = ''
        self.subject.edit_text = ''
        self.message.edit_text = ''
        self.loop.widget = self.frame
    def open_help(self):
        """
        This function displays a nice retro feeling help box onto the screen.
        The information is loaded in via an external text file.
        """
        help_text = [x.replace('\n','') for x in open('docs/help.txt', 'r').readlines()]
        listbox = self.put_text(help_text)
        listbox = urwid.Padding(listbox, 'center', 40)
        listbox = urwid.Filler(listbox, 'top', 15)
        help_box = urwid.Overlay(urwid.AttrWrap(urwid.LineBox(listbox, 'Help'), 'hacker'), self.frame, 'center', 40, 'middle', 15)
        self.loop.widget = help_box
    def unhandled(self, key):
        """
        This functions handles keys which the currently focused widget does not.
        """
        if key == 'f8':
	    """ Allow the user to "Exit" the application cleanly. """
            raise urwid.ExitMainLoop()
        elif key == 'f1':
	    """ Opens the Help box. """
            self.open_help()
        elif key == 'f5':
	    """ Opens the WebConsole so that I can administer the site.  """
            self.app_state = 'TERM'
            raise urwid.ExitMainLoop()
        elif key == 'esc':
	    """ Returns the main view to the frame, this is used to close the help box. """
            self.loop.widget = self.frame
        elif key == 'tab':
	    """ Enable tab navigation between the header and body frames.  """
            if self.frame.get_focus() == 'header':
                self.frame.set_focus('body')
            else:
                self.frame.set_focus('header')
        elif key == 'down':
	    """ Enables the "Down" key to leave the header, useful in the Contact form page. """
            if self.frame.get_focus() == 'header':
                self.frame.set_focus('body')
        elif isinstance(key, tuple):
            if key[0] == 'mouse release':
	        """ This is a workaround for a bug in Shellinabox, where as clickable links
	        don't work when mouse is enabled. """
                if key[3] == 17:
                    self.app_state = 'BITBUCKET'
                    raise urwid.ExitMainLoop()
                elif key[3] == 18:
                    self.app_state = 'PYTHONDIARY'
                    raise urwid.ExitMainLoop()
    def button_press(self, button):
        """
        This function is called when a tab is clicked, and moves the user to a new tab.
        """
        self.current_tab = button.get_label()
        self.frame.header = self.tabs()
        func = getattr(self, 'page_%s' % self.current_tab.lower())
        self.frame.body = func()

try:
    """ This allows us to skip the intro if we come in from an external app. """
    skip_intro = sys.argv[1]
    if skip_intro != '--skip-intro':
        skip_intro = False
except:
    skip_intro = False

if not skip_intro:
    """ The intro splash screens!  GC'd once their done running to converse memory. """
    PythonPowered().main()
    BigText().main()
main = MainScreen()
main.main()
if main.app_state == 'TERM':
    """ This loads up the backend WebConsole to administer components of the site... Shhh, don't tell anyone. """
    os.execl('/usr/bin/python', '', 'cli.py')
elif main.app_state == 'BITBUCKET':
    print "Now leaving IAmKevin.CA, thanks for visiting!\n\n"
    print "https://bitbucket.org/kveroneau"
elif main.app_state == 'PYTHONDIARY':
    print "Now leaving IAmKevin.CA, thanks for visiting!\n\n"
    print "http://www.pythondiary.com/"

There you have it, this should get you started with building your own ncurses based application. Technically, it's not using the ncurses library, it can use it if you change the urwid screen to be urwid.curses_display.Screen class when starting the mainloop.

About Me

My Photo
Names Kevin, hugely into UNIX technologies, not just Linux. I've dabbled with the demons, played with the Sun, and now with the Penguins.




Kevin Veroneau Consulting Services
Do you require the services of a Django contractor? Do you need both a website and hosting services? Perhaps I can help.

If you like what you read, please consider donating to help with hosting costs, and to fund future books to review.

Python Powered | © 2012-2013 Kevin Veroneau