Tuesday, January 31st, 2012

Reading foreign file formats

Sometimes when using Python you may need to load obscure formats. During my time away from Python, I work on indie RPG games using a game engine called OHR.RPG.CE. It has been around since the mid-90s and is still evolving and growing. It was initially built using QBasic for DOS, and some of the file formats have remained the same to keep backwards compatibility. The engine is now built ontop of FreeBasic. I have been pondering about building the engine entirely in Python.

The first task I set out to do was load the graphics format they use for storing backdrops and tilesets. It is saved in an old format called ModeX and was BLOAD'd into memory. Since the project is open source, they provide plenty of internal format information, so I was able to create the following Python code using Pygame to load these image formats:

from struct import *
import pygame
from pygame.locals import *
import sys

pygame.init()
screen = pygame.display.set_mode((320,200),0,8)
pygame.display.set_caption('MXS Loader')

f = open('palettes.bin', 'rb')
hdr = unpack('hh',f.read(4))
pal = []
for i in range(0,256):
  pal.append(unpack('BBB',f.read(3)))
f.close()
screen.set_palette(pal)

peek = Struct('B')
f = open('splash.mxs','rb')
f.seek(0*64000)
for j in range(0,4):
  for y in range(0,200):
    for x in range(j,320,4):
      screen.set_at((x,y), peek.unpack(f.read(1))[0])
f.close()

while 1:
  for event in pygame.event.get():
    if event.type in (QUIT, KEYDOWN):
      sys.exit()
    pygame.display.update()
    pygame.time.delay(100)

The code makes use of the struct module to load structured binary data. The first blocks set-up the display. The next section of code blocks configures the palette. This graphics format is only 256 colors. Finally in the next code block, we render the image. Notice how the palette is in a separate file than the actual image. It is done like this for a good reason, multiple backdrops and tilesets will share the same palettes. In fact, one MXS file noted above can store a large number of individual images, essentially one could create an animation by displaying them all in sequence.

f.seek(0*64000)

This line controls which image in the set to load, swap the 0 with another number.

I really enjoy playing with file formats and seeing what makes them tick to actually displaying the data within them. Since I enjoy this so much, I wrote a Python program to convert an MXS to a PNG file:

from struct import *
from PIL import Image

peek = Struct('B')

im = Image.new("P",(320,200))
f = open('palettes.bin', 'rb')
hdr = unpack('hh',f.read(4))
pal = []
for i in range(0,768):
  pal.append(peek.unpack(f.read(1))[0])
f.close()
im.putpalette(pal)

pix = im.load()
f = open('splash.mxs','rb')
#f.seek(0*64000)
for j in range(0,4):
  for y in range(0,200):
    for x in range(j,320,4):
      pix[x,y] = peek.unpack(f.read(1))[0]
f.close()

im.save("splash.png", "PNG")

This program is very similar to the last one, except it renders the output to a PNG file instead of an SDL surface. Since this was hardly a challenge for me, I thought of rendering the image using pure javascript(Yes I know this is out of context for this blog):

var palFile = new BinFileReader("palettes.bin");
hdr = [palFile.readNumber(2),palFile.readNumber(2)];
pal = [];
for (i=0;i<=255;i++){
  pal[i] = 'rgb('+palFile.readNumber(1)+','+
  palFile.readNumber(1)+','+
  palFile.readNumber(1)+')';
}
var canvas = document.getElementById('mxs');
var ctx = canvas.getContext('2d');
var mxsFile = new BinFileReader("splash.mxs");
for (j=0;j<=3;j++){
  for (y=0;y<=199;y++){
    for (x=j;x<=319;x=x+4){
      ctx.fillStyle = pal[mxsFile.readNumber(1)]
      ctx.fillRect(x*2,y*2,2,2);
    }
  }
}

In order to use this javascript example, you will need to obtain a copy of BinFileReader.js. I was also able to render the file server-side via Django and display it on the browser.

Python Powered | © 2012-2014 Kevin Veroneau