0.4 Release: setting up the scaffolding

The first scheduled release of my project sundae.js for OSD700 will be released for Sunday January 30. In case this is your first time reading my blog, my project is a automated testing framework designed to test WebGL dependencies and functionality in addition to YOUR library using those very WebGL functionalities, that is sundae.js.

0.4

There are main 3 goals for the first major release of sundae.js:

Re-factor framework

The original outline of the project was largely taken from processing.js's developer tools reference tester. However as the scope of my project expanded a similar change had to be reflected in the way the code was designed. In order to facilitate this change the old reference tester has to be ripped into key sections, The test building process has to be clearly defined:

• Select the tests
• Load the tests dependencies
• Build test div and canvases
• Load test into canvas
• Load know good into canvas
• Blur both canvases
• Pixel by pixel compare each canvas
• Generate test result into canvas
• Load next test 
• Compile results

What needs to be learned for this process is how to dynamically load Javascript libraries, and then a method of how to recognize when a dependency is already loaded.

Define API

As the project expands a clearly defined programming interface is necessary to maintain workflow. This design needs to have scalability at the forefront of the decision making process. We know each test case should be treated as its own object, an example of what to expect would be: dependencies, known good, error message, test. Additional standards will be that tests will load into a canvas as will their respective last known good result, from there the rest of the test process is built in.

Define test case structure

The thought process behind this goal is to allow for an easy to implement and expandable way to get your test cases working with sundae as fast as possible. The goal is to use a JSON object structured by my API design to clearly define everything optional and required for each test. Further additions include characterize tests by a directory structure. An obvious skill gap of JSON methodology will need to be bridged, however the benefits will be far reaching.

That is everything set up for the 0.4 release of sundae.js!

• Bug tracking
• Repo
• Wiki

Continuing with OSD: The rematch

Goal
Create an easy to implement multi platform framework with the intent to compare webGL canvases and/or webGL buffers with known good or known bad results. Additionally the focus will be on webGL functionality relevant to c3dl, and c3dl rendered canvases.

What is it
The goal of this reference tester is to provide a reliable and extend-able framework to isolate and test webGL functionality in addition to the libraries which use webGL. From the libraries point of view the aim is to isolate what really broke, to point blame on either specific test cases: using small pieces of unique code or isolated internal functionality of webGL. Reasons for a failure are expected to be broken code or a change specifications in the latest implementation. 


The Plan

Initial Steps
Rebuild the framework with modularity in mind. Clearly define and separate the entire process for sanity and easy of use purposes. Areas of focus: test case import, framework setup, specific test case setup, building test case, building reference, generate result of comparison, compile test results. Using that assumption each of the resources being tested will need its on own builder function to generate a testable piece of data. My current idea is to compare all rendered objects using a canvas checking its pixels against a known solution on the other hand I'm unsure how ill approach comparing buffer objects. Additional work will need to be done to confirm how to isolate and test webGL functionality. 

Direction Afterwards
After rewriting the code into a workable framework and developing the methodology of how the tests are being conducted. The work changes pace and focuses more on rounding out the test case base. The focus should primarily be on isolating webGL, particularly the webGL relevant to the c3dl rendering process. In addition to that attention must be paid to breaking down the c3dl rending components and testing them individually. 

Fine Tuning
Once a working test base has been established and the framework defined the task becomes fine tuning and optimizing the reference tester. Improving the test case input and organization for scalability eg. after selecting the desired test cases the tester should include directories containing those tests and import them in. This allows the test cases to be separated into files and also be organized by directories. Other necessary improvements would be implementing worker threads to enhance run time of the entire ordeal and especially important the tester should be fully multi platform compatible. Having more platforms under the testers umbrella allows known fails created by build bugs on the part of webGL to because better recognized to the library being tested or Mozilla itself.


Releases

• 0.4
• Clearly identify the test process, and approach
• Rewrite framework defining and separating the testing sequence  
• Update readPixels()
• Confirm pixel comparison and blur() actually work
• 0.5
• Implement howto test webGL fundamentals. Both rendered and buffered objects.
• Develop relevant webGL test cases
• Break c3dl tests into more fundamental tests. Isolating very specific objects.
• 0.6
• Port test cases into their own files
• Implement importing test case files
• Define rigid organizational structure, directory and tag based import system. 
• 0.7
• Enhance optimization with worker threads
• Identify locations for multiple threads and strategy of use
• Implement multi thread support
• 0.8
• Implement Windows operating system support with all browsers
• Implement relevant webGL test cases for Windows and specific browsers, including known fails.
• 0.9
• Implement Apple operating system support with all browsers
• Implement relevant webGL test cases for Apple and specific browsers, including known fails.
• 1.0
• Implement Linux operating system support with all browsers
• Implement relevant webGL test cases for Linux and specific browser, including known fails.
• Polish code logic: remove unnecessary runtime

And of course the important link to my Project Wiki!
Annnnnd the project Github

c3dl Ref Tester 0.3

Closing out OSD before extracurricular work sets in. Let me break done the work and the problems I encountered.

Before I begin let me explain what c3dl is. C3dl is a javascript library used to make writing 3D applications easier. Most notably it utilizing webGL for graphic accelerated rendering in your browser which is all the rage now.

Structure (before my upcoming rework) was taken from Processing.js. My goal was to take that test structure rip all signs of processing out of it and mash c3dl in. I've adjusted my scope a bit, currently my goal is to rework the tester so that it simply tests 2 canvases and spits the result into a third canvas. That way any method of rendering into a canvas can be used because a pixel by pixel comparison won't care how the pixels got there. Of course my contribution will inject c3dl into one canvas and an image into the other.

The purpose behind putting an image and a c3dl rendered scene is this. The reference images were taken at a point of time where that particular scene rendered properly. Comparing that to the same scene rendered now as you load the test. The idea is any changes that broke or changed the result of c3dl rendering would be apparent.

Some problems I encountered were:

• Wasted 2 weeks trying to generate the reference images. Until I realized you can either canvas.getPNG or even better right click "save as" and get a png.
• Learning c3dl, adding multiple canvases and multiple callback mains for each test case revealed many bugs, learning not to test everyone else code was a huge hurtle.

Future problems and goals:

• Break apart the tester and make it focus more on just the pixel arrays of canvases, not how the canvases were filled.
• Change the test case library to something friendly of a larger scale, based on feedback from David Humphrey. That I have no clue.
• Confirm that the pixel comparison and blurring actually work, so I can trust there results. It seems a bit questionable at the moment but it's out of my current abilities.

Links:

• Matrix hosting
• http://matrix.senecac.on.ca/~cwdesautels/osd600/

• Github Repo
• https://github.com/cwdesautels/c3dl/tree/0.2

c3dl Ref Tester 0.1

What is C3DL?
The Canvas 3D JS Libary (C3DL) is a JavaScript library that will make it easier to write 3D applications using WebGL. It provides a set of math, scene, and 3d object classes that makes WebGL more accessible for developers that want to develop 3D content in browser but do not want to have to deal in depth with the 3D math needed to make it work. Website found here.

Ref Tester

Largely ported from David Humphreys Processing.js ref tester found here. I attempted strip out processing and replace it with c3dl to do similar pixel comparisons between canvas's. This proved far more complicated then I anticipated, i had a lot of trouble interpreting all open source projects I encountered. This was solved through exhaustive scanning of Mozilla Development Network and project specific documentation, in addition to asking the pros on #moznet!

0.1

Numerous errors in implementation can be found here, however 0.1 contains:

• ported Processing.js ref tester
• integration of a single c3dl demo
• blurring and comparison of the canvas's 

Beyond

look forward to a fast 0.2 release pending the reviews requested on current release, future goals include:

• exhaustive still tests
• modularization of tests

Links

• Wiki
• Github
• Lighthouse