Tuesday, January 25, 2011

0.4 Release: setting up the scaffolding

The first scheduled release of my project sundae.js for OSD700 will be released for Sunday January 30. In case this is your first time reading my blog, my project is a automated testing framework designed to test WebGL dependencies and functionality in addition to YOUR library using those very WebGL functionalities, that is sundae.js.

0.4

There are main 3 goals for the first major release of sundae.js:

Re-factor framework

The original outline of the project was largely taken from processing.js's developer tools reference tester. However as the scope of my project expanded a similar change had to be reflected in the way the code was designed. In order to facilitate this change the old reference tester has to be ripped into key sections, The test building process has to be clearly defined:

  • Select the tests
  • Load the tests dependencies
  • Build test div and canvases
  • Load test into canvas
  • Load know good into canvas
  • Blur both canvases
  • Pixel by pixel compare each canvas
  • Generate test result into canvas
  • Load next test 
  • Compile results


What needs to be learned for this process is how to dynamically load Javascript libraries, and then a method of how to recognize when a dependency is already loaded.

Define API

As the project expands a clearly defined programming interface is necessary to maintain workflow. This design needs to have scalability at the forefront of the decision making process. We know each test case should be treated as its own object, an example of what to expect would be: dependencies, known good, error message, test. Additional standards will be that tests will load into a canvas as will their respective last known good result, from there the rest of the test process is built in.

Define test case structure

The thought process behind this goal is to allow for an easy to implement and expandable way to get your test cases working with sundae as fast as possible. The goal is to use a JSON object structured by my API design to clearly define everything optional and required for each test. Further additions include characterize tests by a directory structure. An obvious skill gap of JSON methodology will need to be bridged, however the benefits will be far reaching.

That is everything set up for the 0.4 release of sundae.js!

Wednesday, January 12, 2011

Continuing with OSD: The rematch

Goal
Create an easy to implement multi platform framework with the intent to compare webGL canvases and/or webGL buffers with known good or known bad results. Additionally the focus will be on webGL functionality relevant to c3dl, and c3dl rendered canvases.

What is it
The goal of this reference tester is to provide a reliable and extend-able framework to isolate and test webGL functionality in addition to the libraries which use webGL. From the libraries point of view the aim is to isolate what really broke, to point blame on either specific test cases: using small pieces of unique code or isolated internal functionality of webGL. Reasons for a failure are expected to be broken code or a change specifications in the latest implementation. 


The Plan

Initial Steps
Rebuild the framework with modularity in mind. Clearly define and separate the entire process for sanity and easy of use purposes. Areas of focus: test case import, framework setup, specific test case setup, building test case, building reference, generate result of comparison, compile test results. Using that assumption each of the resources being tested will need its on own builder function to generate a testable piece of data. My current idea is to compare all rendered objects using a canvas checking its pixels against a known solution on the other hand I'm unsure how ill approach comparing buffer objects. Additional work will need to be done to confirm how to isolate and test webGL functionality. 

Direction Afterwards
After rewriting the code into a workable framework and developing the methodology of how the tests are being conducted. The work changes pace and focuses more on rounding out the test case base. The focus should primarily be on isolating webGL, particularly the webGL relevant to the c3dl rendering process. In addition to that attention must be paid to breaking down the c3dl rending components and testing them individually. 

Fine Tuning
Once a working test base has been established and the framework defined the task becomes fine tuning and optimizing the reference tester. Improving the test case input and organization for scalability eg. after selecting the desired test cases the tester should include directories containing those tests and import them in. This allows the test cases to be separated into files and also be organized by directories. Other necessary improvements would be implementing worker threads to enhance run time of the entire ordeal and especially important the tester should be fully multi platform compatible. Having more platforms under the testers umbrella allows known fails created by build bugs on the part of webGL to because better recognized to the library being tested or Mozilla itself.


Releases


  • 0.4
    • Clearly identify the test process, and approach
    • Rewrite framework defining and separating the testing sequence  
    • Update readPixels()
    • Confirm pixel comparison and blur() actually work
  • 0.5
    • Implement howto test webGL fundamentals. Both rendered and buffered objects.
    • Develop relevant webGL test cases
    • Break c3dl tests into more fundamental tests. Isolating very specific objects.
  • 0.6
    • Port test cases into their own files
    • Implement importing test case files
    • Define rigid organizational structure, directory and tag based import system. 
  • 0.7
    • Enhance optimization with worker threads
    • Identify locations for multiple threads and strategy of use
    • Implement multi thread support
  • 0.8
    • Implement Windows operating system support with all browsers
    • Implement relevant webGL test cases for Windows and specific browsers, including known fails.
  • 0.9
    • Implement Apple operating system support with all browsers
    • Implement relevant webGL test cases for Apple and specific browsers, including known fails.
  • 1.0
    • Implement Linux operating system support with all browsers
    • Implement relevant webGL test cases for Linux and specific browser, including known fails.
    • Polish code logic: remove unnecessary runtime
And of course the important link to my Project Wiki!
Annnnnd the project Github