Inside Demoscene Pinball
This page explains the technology behind the game Demoscene Pinball. It gives a deeper insight to the different features of the game than what is observable by just playing it.
Table of Contents
- Glossy metal material
- Sharp metal material
- Cube map reflections
- Table surface
- Glass borders
- Background environment
- Billboard chains
- Score display
- Ball lock
- Java2D shapes
- Amiga Ball
- Physics visualization
- Resource management
- Filesystem watcher
- Hotswap Java agent
- Sounds and music
- Online highscores
- Video capture
Demoscene Pinball was created for Assembly 2014 Game development competition and was voted for the 4th place. It was in development for three years with slow but steady progress.
The game is written in Java 1.7. It uses LWJGL (The Lightweight Java Game Library), which provides Java Native Interface for OpenGL, OpenAL and input. The code base is 25 000 lines of Java code divided into 600 classes in 300 files.
Native Windows executable and installer was created with Excelsior Jet 10.0.
The game uses a custom rendering engine using OpenGL 3.3 Core. The rendering engine was originally based on my previous game Ufohippa 3 (http://www.pouet.net/prod.php?which=55532), but no longer shares much common code.
Instead of the traditional Phong lighting model, the game features several less used techniques in the rendering. These give the game a unique visual style. The different rendering techniques and related shaders are explained below.
The most important material in the game is glossy/brushed metal which is used all around the game table. This is a powerful shader implementing a sort of global illumination, where all lighting in the game is reflected from many vertical surfaces. It's powerful in the sense that it gives colorful detail to the surfaces without the need for textures and does not need much configuration for lighting. It also improves the gameplay, as the player can track the movement of the ball from the glossy reflections even when the ball itself is hidden behind other objects.
The glossy metal (brushed metal) material in the game consists of two different calculations. One formula for horizontal and another one for vertical reflections are combined depending on the Y-component of the surface normal. As the game contains mostly fully horizontal or fully vertical polygons, the factor is usually 0 or 1, but the transition between them is still smooth. Most of the game objects use this shader.
Vertical reflections are achieved with an image based lighting approach. Once per frame the table is rendered to a texture from above with an orthographic camera and blurred slightly. The rendering tightly covers the whole bounds of the table surface. The fragment shader calculates a reflection vector based on the normal and projects it onto the XZ-plane. Then N samples are taken from the screenshot starting from the location below the shaded pixel and iterated in the direction of the projected reflection vector. The vertical component of the reflection vector also slightly affects the sampling offsets. In the game N=20 provides good enough quality. A texture consisting of horizontal stripes is also applied to the formula.
In the first version glossy metal reflections for horizontal surfaces were implemented with anisotropic lighting. Anisotropic lighting is needed for brushed metal, where the roughness of the material is sharp in one direction and glossy in the other one. First approach was to provide tangents and bitangents in the vertex data and to calculate lighting based on those. This didn't work out because I wanted a circular pattern which is not possible to produce by linear interpolation from vertex data. Therefore the tangents and bitangents are now calculated in the fragment shader from a single uniform specifying the center of the circular pattern. This proved out to be a very good choice, as now none of the meshes need tangents or bitangents. Only the single vector uniform is needed per mesh. The limitation is that this solution only works properly for meshes formed by extruding 2D shapes, but that is fine in this game.
Later when directional light sources were replaced with environment map lighting, this shader calculating anisotropic lighting was also updated. Instead of calculating how much the light sources affect along the reflection vector projected to the bitangent, the shader now takes N samples from a blurred version of the environment map. The samples form a semicircle in the plane defined by the surface normal and bitangent. The number of samples is 64 and they are distributed evenly.
Sharp reflections follow an idea similar to the vertical glossy reflections. Once per frame a single screenshot of the table from above is taken, but not blurred. The fragment shader finds the intersection of the reflection vector and the table surface and samples the screenshot from this location. Although not exact, in practice in this game the effect is good enough. To make the edges shinier, the reflection calculations include a Fresnel term calculated with Schlick's approximation.
Most of the clear shiny objects in the game use this shader. The effect is most visible in the spinner and in the metal rails.
The ball has its own shader utilizing dynamic cube map reflections. The scene is rendered to each of the six faces of a cube map using cameras located at the center of the ball with 45° field of view. The shader simply samples from the cube map with the reflection vector. In addition to that, a Fresnel term makes the edges shinier producing better depth effect.
Only one cube map reflection is rendered per frame to keep the framerate consistent. During multiball the game uses round robin to select one of the active balls and the remaining balls are rendered with the reflection rendered during previous frames.
The table surface is a unique object with support for reflections, simple ambient occlusion and dynamic emission map.
When rendering the table surface, the table quad is first rendered to a stencil buffer. Then the whole scene is rendered again, reflected with a mirror matrix, in lower detail and inverted culling to produce the reflections on the table. The reflection is also clipped with the stencil buffer inside the table border. The Fresnel term is important to emphasize the reflections in low angles in this shader.
A less visible feature of the table surface is ambient occlusion from the balls. Ambient occlusion is calculated analytically and supports all three balls during multiball. Compared to the screen space ambient occlusions seen in modern games, this approach is more realistic, but is limited to simple geometry such as spheres.
Emissive table lights
To support togglable and blinking lights the three textures are passed to the shader. The first texture is the traditional color texture which is used when no lights on the table are on. The second texture specifies the actual lights that can be enabled. These are mostly the same colors as in the first texture where lights are present and black elsewhere. This is additively blended on top of the first texture, but conditionally for each fragment. The third texture specifies regions for each light. An array of booleans packed to a single uvec4 is passed as a uniform to the shader to specify which areas are lit. When a fragment is shaded, the shader reads from the third texture on which light area the fragment belongs to and then checks if that area is enabled in the boolean array. If yes, then the value from the emission map is added to the base color.
Glass borders use a simple trick to achieve order independent transparency. Normal alpha blending with the formula source.alpha*source.rgb + (1-source.alpha)*destination.rgb does not play nicely when rendering in arbitrary order. On the other hand rendering concave objects in back-to-front order is far from trivial. Fortunately there are few blend modes which don't care about the ordering. One is additive source.alpha*source.rgb + destination.rgb and another one is multiplicative source.rgb*destination.rgb. Typically the additive blending mode works well for particles that emit light, but in this case it is used to simultaneously add and subtract different color channels. This shifts the color of the pixels behind glass towards specified target color without making them brighter. E.g. in the screenshot below the green color value is increased while red and blue color values are decreased. If the additions and subtractions are in balance, the effect can be pretty convincing.
To further improve the glassy appearance, the thickness of the glass is also simulated by approximating the distance that the light ray traverses through the border. An exponential formula is used to apply more colorization as the distance grows. The approximation is calculated based on the light direction, surface normal and a vertex specific factor.
The background behind the pinball table is a cube map created from a stitched panorama picture. The individual pictures were taken from Hartwall Arena during Assembly 2013 with an ordinary digital camera. Unfortunately when the game was presented in Assembly 2014, the demo party had moved to Helsinki Fair Center. A talented artist could have created a lot better background.
The skybox is rendered with a single fullscreen quad. The eye vector is transformed by the inverse of the camera matrix and used as the sampling direction from the cube map.
Normal map diffuse lighting
All diffuse lighting in the game is calculated from the skybox. A heavily blurred version of the cube map is sampled from the surface normal direction in each shader that supports diffuse lighting. This allows easy matching between overall lighting and background environment colors.
The game achieves HDR by rendering everything to a floating point off-screen frame buffer. Then all bright values are extracted to another frame buffer and blurred with Gaussian blur. The blurred version is blended on top of the original rendering. Finally the end result, which is still floating point colors, is tone mapped to RGB8 format. The tone mapping formula is a custom one that first scales the brightness of a color to [0, 1] range while preserving saturation. After that it converts very bright values towards pure white.
The engine supports billboard chains, continuous connected shapes in 3D that are automatically seamlessly oriented towards the camera. This was to be utilized for example in neon light text objects around the table. Eventually the whole feature ended up not being used, but the code still exists in case it is later needed. The orientation of the polygons is calculated in the vertex shader based on mesh's vertex attributes specifying the chain direction.
The pinball table has a virtual 64x16 pixel full color display on top of it. There are a few fonts of different size for printing text on it. The display also features an animation system and a possibility to display a combination of arbitrary pixel based effects and vector graphics drawn using Java2D on it. The shape of a "pixel" is specified with a texture.
All 3D objects in the game, excluding the Duck, are code generated. No 3D modeling software was used for creating this game. The different object types are explained below.
Metal rails are the most complex objects in the game. The basic building block is a 3D spline which specifies the path of the rail. A rail generator automatically creates curved metal pipes around the spline. The generator can be configured to create either half pipes or full pipes for any section in the spline. It also allows specifying where support arcs should be placed. In addition the ending shape of the rail can be configured. Junctions and crossovers are also supported by specifying the spline curve of the intersecting rails, in which case the generator cuts some of the metal pipes from the intersecting locations. The exact cutting points are calculated numerically, so it's not limited to the tessellation detail. In addition in every place where a metal pipe starts or ends, a hemisphere is generated to close the pipe.
The ball lock is a special type of rail mesh that can store up to three balls. When the third ball enters the lock, the object starts to rotate. The shape of the lock rail is designed in a way that when it rotates, the gravity pulls the balls out from the lock one by one. After all balls have been released, it rotates back to the resting orientation.
The plunger is barely visible in the game due to the limited camera angles, but the implementation is still interesting. First of all, the physics of the plunger uses RK4 integration for the damped spring equation, which gives stable behavior with varying framerates and parameters in this game.
Second, the geometry of the spring is calculated in a vertex shader. This allows compressing the spring without affecting the thickness of the spirale, which would be the case with ordinary scaling. The mesh is pre-generated for the proper vertex count, but the positions of the vertices are calculated in the vertex shader based on information stored in the vertex attributes and a single uniform specifying the spring tension.
There is no special object type for ramps. They are just generated meshes that form a 3D ramp shape for the specific purposes. There are two different ramps in the table.
A major part of the geometry in the game is generated using Java2D Shapes and extured to 3D. Java2D also provides a powerful Constructive Area Geometry solution, which allows combining and carving the shapes to desired forms. The tessellation can also be controlled and different tessellation levels are used for different purposes.
There are three bumpers on the pinball table positioned in a cluster. The bumpers have an invisible trigger which detects when a ball hits them. When that happens, the ball is given an impulse in the direction of the collision normal. The hit it visualized by animating a vertically moving ring and a fast light blink implemented with emissive material. There is also a cool-off period to not trigger the same bumper again too fast. The texture on top of the bumpers is the first texture implemented in the game and has been generated with Java2D.
The shape of the flipper is generated by creating two arcs and connecting them with lines. A whitish rubber band is generated around the shape for visualization. The physics of the flippers is implemented as a kinematic object in Bullet physics and a hinge constraint controlled with an angular motor.
The geometry of the Amiga ball is almost like a simple geosphere with low tessellation. The red/white coloring is controlled by vertex attributes. In addition few of the faces are detached from the rest of the ball and can be rotated around the center of the ball acting as a gate that opens and closes the access to the ball. The backside is also open without any gate. An invisible trigger inside the ball detects when a pinball ball enters the Amiga ball and traps it for a while.
The duck is the only 3D object in the game that has not been generated procedurally. The vertex and polygon information of the 3D model was simply converted to Java arrays. Despite its simplicity, it's an important part of the scoring mechanism in the game. Also it says "quack" when the ball hits it.
The spinner is a metal plate with an eccentric hinge constraint. When a ball hits the spinner, it applies an angular impulse for it causing it to rotate. Whenever the spinner rotates a full revolution, it generates an event and gives points for the player.
Slingshots are vertically extruded Java2D shapes surrounded with the same rubber bands as in flippers. In addition there is a physics detector, which gives the ball an extra impulse in the direction of the collision normal when colliding on specific side of the slingshot. There is also a tolerance based on the ball velocity in the normal direction.
There are nine buttons scattered around the table with letters forming the word DEMOCOMPO. When a button is hit, the corresponding light is lit and the button gets translated towards the wall holding it. The player must hit all the buttons to collect demo effects. The buttons are formed of two boxes. When a button is created in the code, the corresponding table light is also generated automatically in front of it. The buttons only detect ball collisions without giving any collision response back to the ball. This makes it easier to hit multiple buttons in a row vertically.
There is a group of three rollovers with letters GRZ. When the player has enabled all three letters, he collects a "greeting". Each greeting awards more points than the previous one and collecting all six greetings awards an extra ball. One physics trigger is placed for each of the three lights and wider triggers are placed on top of and below them. A rollover is lit when the ball collides with the bottom, middle and top triggers in either order. Player can also shift the lit rollovers with the same keys that control the flippers.
Demoscene Pinball uses JBullet (http://jbullet.advel.cz/) for its physics engine. Since the game is very fast paced, the physics is updated 180 times per second. Other logic in the game is updated 60 times per second to match the refresh rates of typical monitors. The following features from the physics engine are used:
- Static, dynamic and kinematic objects
- Sphere geometry using SphereShape
- Cylinder geometry using CylinderShape
- Convex mesh geometry using ConvexHullShape
- Concave mesh geometry using BvhTriangleMeshShape for static objects
- HingeConstraint for objects constrained to rotate around a fixed axis (flipper, spinner and lock rail)
- DiscreteDynamicsWorld with DbvtBroadphase
- Invisible hit test objects without collision response
- External impulses
JBullet (or the original Bullet Physics) does not currently provide deterministic serialization to allow saving and restoring the physics state. Due to this the game resets and recreates the physics state every frame. The objects and constraints are not recreated, but they are re-added to a newly created physics world every frame. While not exposed to the end users, the game engine allows saving and restoring game states or creating in-game cut-scenes with prerecorded gameplay. The gameplay is recorded by saving the whole keyboard state to a single byte each frame into a compressed stream.
Triggers are invisible physics objects which do not provide collision response. The shape of the triggers can be anything. They just detect when a collision happens with the ball and report it to the logic engine. There are three types of triggers in the game. A continuous triggers reports a collision every frame a ball intersects it. A non-continuous trigger reports a collision only once when a ball intersects it. It can be triggered again only after the ball has first left the trigger. The last trigger type is a trigger sequence consisting of arbitrary triggers. The trigger reports a collision when all its child triggers are activated in order. If a trigger out of order is detected, the whole sequence resets and must be started over. The sequence can be configured to also work in reverse direction.
The graphics engine has a Level Of Detail system which allows selecting different geometries and materials depending on the rendering settings and context. One of the detail levels is dedicated to physics, which allows visualizing the game from the perspective of the physics engine. Each object specifies geometry for the physics visualization, which is typically tessellated with lower detail than the visual one. This helps development and debugging.
All resources (meshes, shaders and different types of textures) are specified through generators. The generators are registered to a resource manager, which controls loading (and reloading) of the resources. It also ensures that no resource is loaded more than once. The resource manager provides convenience methods for creating typical resources such as image textures or shaders from glsl files.
The resource manager also provides a Swing based menu for selecting a texture for preview. The texture is rendered as a single 2D quad or multiple quads in the case of cube maps. This feature is handy for debugging generated or render target textures, which most of the textures in the game are.
Resources can have arbitrary number of dependencies, for which the resource manager installs a listener. When the dependency is changed, the resource manager triggers reloading of the resource depending on it. The game uses Java's FileSystem.WatchService to monitor changes in the filesystem where resource data is loaded from. This allows realtime updating of textures when the texture image is modified on the disk with image manipulation software. The same applies to shaders when the shader source code has been modified.
A more interesting dependency is ClassDependency. The game comes with a Java agent implementation that hooks to the debugger and listens for class "hotswapping". When Java applications are run with a debugger attached, classes can be recompiled and patched to the virtual machine on the fly. Demoscene Pinball can detect when this happens and trigger reloading of resources that have been generated with the changed classes. This allows near realtime updating of meshes and textures which are procedurally generated.
The last supported dependency type is one that triggers reloading when the display size has changed. This is used for some render target textures.
Sounds in the game are implemented with OpenAL, for which LWJGL has a JNI wrapper in addition to OpenGL. Voices have been generated with an online speech synthesizer. All other sound effects are extracted from a recording of actual pinball gameplay. Sound editing was done with Audacity.
The background music was created with cgMusic (http://codeminion.com/blogs/maciek/2008/05/cgmusic-computers-create-music). The software generates pseudorandom MIDI files. The result was converted to Ogg Vorbis format using SynthFont (http://www.synthfont.com/Downloads.html) with a high quality sound font called CrisisGeneralMidi from http://www.bismutnetwork.com/04CrisisGeneralMidi/Soundfont3.0.php.
In addition to the normal background music, there is a variation of it which is played during different special modes. The variation was created with the same cgMusic software but with a different arrangement and converted to MIDI with a different sound font called GenesisSF (http://www.cinos.biz/old-cinos/genesisf.html). Because both music files have the same tempo and structure, the game can fade between them very smoothly.
Because OpenAL can only play raw sound data, OggInputStream from http://home.halden.net/tombr/ogg/ogg.html is used for decoding Ogg Vorbis files, and a self-made music player is used to control the playback and stream the data to OpenAL.
The game features online highscores, where a database of all submitted highscores is stored on a server and accessed by the client. The client communicates with the server using the HTTP protocol. The server side was implemented with PHP due to simplicity. The highscore data is stored in a MySQL database. All network connection on the client side is executed using Java Futures and a thread pool, which prevents blocking of the user interface while communicating with the server.
The selected choice of server side technology also allows using the same backend code to publish highscore results on the website of the game.
The highscores and the rest of the menu system are presented using a simple custom bitmap OpenGL font implementation, where the bitmap font data is created with BMFont (http://www.angelcode.com/products/bmfont/) and rendered with one quad per glyph.
At the time of writing this page, three months after the release of the game, over 2300 highscore entries have been submitted to the server.
The development version of Demoscene Pinball can record video captures of the game using Xuggler library (http://www.xuggle.com/xuggler/). Due to licensing of the library this feature is not available in the release version. The introduction video (available from http://demoscenepinball.dy.fi/) was created using this in-game recording. The game uses a background thread and OpenGL Pixel Buffer Objects to transfer the rendered image frames to RAM and encodes them to high definition H.264 video format in real-time with little overhead. In addition the video recording captures the audio data that is going to the speakers and encodes that to the video as well. As an end result it is very easy to create high quality video recordings of the game without any need for external software.
This page was last modified on 5.2.2015