Showing posts with label blackberry. Show all posts
Showing posts with label blackberry. Show all posts

Thursday, August 4, 2011

BerrySync: Fetcher.java, now with more github

after a little bit of work ironing out the BerrySync app. Ive shlapped together my synchronization service library. I've dubbed it fetcher.java, now on github.


My Design

I created the library to be as modular as I could, I'm not happy with the results yet but this is the first step after all.

The Fetcher class is a singleton and in order to be used it requires a service to be set, this service must be of the iFetcher interface. That allows me to hotswap or customize the service that I'm using.

On top of that each service can have a different structure or set of needs, so attached to the iFetcher interface are setters for the seperate modules:
  • Encryption module
  • Json decoding module
  • Network connector module

This Release

 I've implemented the bare minimum required steps for FireFox's Sync service. My chosen platform was BlackBerry(obviously), and as a result the modules contain platform specific code. Meaning that different modules will need to be created to port this service to different platforms.


What Can I Do?

Fetcher.getInstance().setLogin("asdf","asdf","asdf").pull();
< This screenshot is the alpha visual representation of data pulled by Fetcher >
1,  2.




Thursday, July 21, 2011

BerrySync: Untested code is broken code

TLDR - What did I learn?

In the irc channels I lurk, a phrase reigns true. "Untested code is broken code". I overlooked this bug because my tests didn't keep up with the code. Don't be lazy, you will not remember you took short cuts a month ago.



BerrySync's Show Stopper
 
So I'm very quick to point fingers, So naturally when our program stops decrypting 75% of the I went straight to http://bit.ly/r9vKFS. You should note RIM is abiding by this document http://bit.ly/ey4Fg and if you read further down you'll note that this algorithm padding standard does not mention AES encryption which is tragic because its the only symmetric key unformatter engine that RIM supplies, meaning you'll need it for AES encryption! For example BerrySync's crypto module sans error control:

AESKey key = new AESKey(keyData);
InitializationVector iv = new InitializationVector(ivData);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
BlockEncryptor encryptor = new BlockEncryptor(
     new PKCS5FormatterEngine(
          new AESCBCEncryptorEngine(key, iv)), outputStream);
encryptor.write(plainText);
encryptor.close();
outputStream.close();
return outputStream.toByteArray(); 

I And you should check out http://bit.ly/rplM0s. Here you'll see that Mozilla is abiding to http://bit.ly/piPqcJ. The 2.1 standard includes AES encryption in the spec, hold on a minute. RIM must be doing something totally off the wall here. All of that combined with the knowledge that using third party and unlicensed encryption and getting your app into AppWorld would involve many headaches and much money ala ECCN.



Wrong

As with most things my finger pointing was wrong, the bug actually lived on this line
data = data.replace('-', '\0'); 
Where my lazy attempt at removing characters from a string totally backfired. The interesting thing is that during rare cases, my sync account and about 25% of others tested it didn't actually make a difference and everything worked as intended. Where the other 75% of accounts saw a Bad Padding Exception during decryption.

Wednesday, July 6, 2011

BerrySync: How fetcher gets it done, thread pooling

A Little Bit Of Background
I designed My fetcher library to have modules, these house the logic for the various steps I would need to perform. The fetcher itself links up the modules and goes from a url to a usable object for BerrySync.

One of the more troublesome modules was the Smuggler, I originally designed smuggler to do the network scheduling for fetcher then Smugglers worker thread would perform the actual connections, making the operation non-blocking(which is important for a good user experience!).

Now
I was forced to change the design, because the data parsing and decryption had to moved off the main thread. To facilitate this I gave Fetcher its own internal work Queue, with the smuggler still scheduling the network connections this time using Fetchers new and improved .invokeLater(runnable r) method!


public void invokeLater(Runnable work){
_queue.enQueue(work);
}
private final class WorkQueue {
        private Node _head;
        private Node _tail;
        private QueueWorker[] _worker;
        public WorkQueue(){...} 
        public synchronized void enQueue(Runnable data) {...}
        private synchronized Runnable deQueue() 
throws InterruptedException{...}
        private final class QueueWorker extends Thread {
            public void run() {
                Runnable work = null;
                while (true) {
                    try {
                        work = deQueue();
                        work.run();
                    }
                    catch (InterruptedException ignored) {}
                }
            }
        }
    } 

I'm using the runnable interface to formalize how this thread can 'run' your work. The commented out methods just show a simple queue linked list data structure.

Tune In Next Time
In the next instalment I'll tell you all about how I'm tracking down an elusive decryption bug with sync data on the BlackBerry!

Wednesday, June 22, 2011

BerrySync: Launching Opera mini xor BlackBerry's Native Browser

An important part of BerrySync will be actually opening up URLs in the browser(s) on your BlackBerry.
After doing a little research I worked out how to not only open the native browser but third party browsers as well.

Opening Native Browser

browserSession = Browser.getDefaultSession();
browserSession.displayPage("http://www.BlackBerry.com");
browserSession.showBrowser(); 

Wasn't that painless and easy?

Opening 3rd Party Browsers, Opera Mini in this case
Also worthy of note is that fact that this block of code doesn't stop at just browser, you can invoke any third party app by replacing "Opera Mini" or making another if clause. See below or Pastebin.

int[] apps = CodeModuleManager.getModuleHandles();
ApplicationDescriptor[] ad;
ApplicationDescriptor holder;
String[] args = {"google.com"};
for(int i = 0, len = apps.length; i < len; i ++){
    ad = CodeModuleManager.getApplicationDescriptors(apps[i]);
    if(ad != null){
        for(int j = 0, adlen = ad.length; j < adlen; j++){
            if(ad[j].getName().equals("Opera Mini")){
                holder = new ApplicationDescriptor(ad[j], args);
                try {
                    ApplicationManager.getApplicationManager().runApplication(holder);
                    break;    //Needed
                } catch (ApplicationManagerException failedToOpen) {}
            }
        }
    }
} 


Code Walkthrough
This code scans through all installed modules on a BlackBerry, it tries to get the application descriptor on each and verifies it against the name of the app you wish to run. The args step is optional, but for our purposes we want to open Opera and goto the specified URL. Lastly the break is needed to stop multiple instances of the app opening since I found that many apps had more then one matching application descriptor

Wednesday, June 15, 2011

BerrySync: How does Firefox Sync use cryptography

I have a blog post explaining how to get FireFox Sync JSON, just having that encrypted JSON isn't enough. So what now?

Sync's Cryptography

When we started work, I used this blog to identify and determine how sync was using expecting it's clients to implement cryptography.

However some issues come along with not understanding cryptography and having never had to use it before, Sync updated and simplified their model in 1.6 and only today with a lot of help from moznet's #sync channel I got a usable understanding of the implementation. Here goes it:

Decrypting the Sync JSON:

There are a few assumptions in you should expect to handle Sync's data.

All of the keys are symmetric which means, one key that can perform both encryption and decryption. These keys are 256 Bit AESKeys used in a mode that requires Initialization Vectors.

Additionally all of Sync's keys have a paired hmac and this is used to verify what you're decrypting is what was encrypted so you should verify each object before decrypting it.

And lastly all JSON received from the server is Base64 encoded and needs to be decoded to be used.
  1. Generate the decryption key's bytes and its hmac bytes from the account's sync key
  2. Create a 256 bit AES Key from the bytes in step 1, along with the Initialization Vector on the crypto/keys object
  3. Decrypt the crypto/keys cipher with the key made in step 2. This reveals a base64 encoded key and hmac pair
  4. Assuming you have a users tab collection object (from a previous post), use the key bytes from step 3. and the specific tab object's Initialization Vector to create another 256 bit AES Key
  5. Decrypt the tab objects payload with the key made in step 4.
  6. Profit!

BerrySync: Firefox Sync's JSON urls

I've been working on BerrySync for just over a month now. We've gotten a lot closer to obtaining usable Firefox Sync data, just an update: yesterday I implemented Sync's credential expectations, and last week we determined the url semantics and got a trivial HTTP connector with a work queue implemented on the BlackBerry. 

Getting the Sync JSON:

  1. With the user's account info, create their sync username: http://bit.ly/mkoeRx
    1.  If it's an old sync account there account name becomes the username
    2. New accounts are emails, which need:
      1. Lowercase it
      2. SHA1 digest it
      3. Base32 encode it
      4. Lowercase it again
  2. GET request http://auth.services.mozilla.com/user/1.0/syncUsername/node/weave
    1. This returns your weave server's address.
  3. GET request weaveServer/1.1/syncUsername/storage/crypto/keys
    • Example Url from Sync 1.1 API
    • All weave node requests will require your syncUsername and account password for authentication
    • This returns a JSON specified here
So I have this implemented in our application BerrySync, currently it's encrusted with RIM libraries and not a library itself, just a workable class that contains a stack of url's which a worker thread plows through returning the results asynchronously.

Giving Back

I intend to package this HTTP connector built in Java as an open source library, hopefully it will contribute to Firefox Sync's third party ecosystem!

Tuesday, June 7, 2011

BerrySync - Unit Testing in Blackberry Apps

Pains
Building Berrysync from the ground up hasn't come without it's challenges. Most notably as of recent has been our incorporation of test driven development, for many people that brings JUnit to mind. To my surprise JUnit without including its dependencies will not natively run on the BlackBerry Java core, and even more interesting is the fact that JUnit comes bundled in the BlackBerry Developer Eclipse!

Thanks to some googling and blog posts lost in my browser history I came across this awesome tool called BUnit. It seems to be the only reliable way to unit test inside a BlackBerry app.

Using BUnit in Your Project
You'll need to define an alternate entry point to your application and add some logic to the App's entry point, here are the steps:
  1. In BlackBerry_App_Descriptor.xml -> click Alternate entry points
  2. Click add provide a name, click ok
  3. Click on what you just added and provide an Application Argument and/or an icon
  4. Open MyApp, and provide this logic (eg. Application Argument = bunit)

    public static void main(String[] args){
        if (args != null && args.length > 0 && args[0].equals("bunit")){
            TestRunner tester = new TestRunner();
            tester.enterEventDispatcher();
        }else{
            MyApp theApp = new MyApp();       
            theApp.enterEventDispatcher();
        }
    }




Two entry points allows this, where BerrySync is our application and B-unit is the unit tester


Tips On Usage
Now I assume the limitations of BUnit and the major benefits of JUnit come from java Reflection. Which you guessed it, isn't included in the BlackBerry JRE. Sooo much to my disappointment in each test suite you need to manually specify the number of tests, and later on invoke them in a switch. On top of that you'll need to add your suites to runner manually.

A snippet of my test suite: http://pastebin.com/mq9QzpkW


I also had to add a few Assertion functions, graciously BUnit provided the file to place them in.

assertEquals(String test, byte[] expected, byte[] actual)
assertNotEquals(String test, byte[] expected, byte[] actual)


here they are too! http://pastebin.com/H3R69XC6

ScreenShot
<- Berrysync's unit tester running as is

Friday, May 13, 2011

BerrySync - Fifefox sync crypto, a step closer

BerrySync
To make a BerrySync compatible with FireFox Sync we need understand how sync connects to its servers. There are two challenges for me so far. First I opted to emulate sync's encryption steps  on the BlackBerry in Java, next I need to make a make a custom sync server and start talking to it before I start talking to the real thing.


FireFox Sync Crypto
So in order to even make sense of the data I'm getting back from the Sync Server I need to know what to do with it! I'm going to break down how I've understand Sync is expecting this to be handled.

  • When you sign up Sync makes a RSA 2048 bit key pair, that's whats used to encrypt during travel, I've been doing my reading here.
  • Decrypting
    1. Decrypt Weave Object with your private key
    2. Decode Base64 Weave payload to binary
    3. Decrypt payload binary with AES 256 bit key and 16 bit Initialization Vector, these are found in the Weave object
    4. Profit from here
Normally you would use Sync's client JavaScript to take care of this, however I attempted to port this to Java on the BlackBerry. Check out my encryption equivalent attempt of steps 2 and 3 here.Thankfully these algorithms are standardized and RIM's crypto library has had exactly what I've needed to far. I still need to confirm how BerrySync is going to handle the RSA keypair.


FireFox Sync Server
I'm at the point where I should start looking for my own custom firefox sync server, Seneca may have one. I was also going to set up one on my laptop this weekend using these:


So basically my goal for next Friday is to have access to a custom FireFox Sync Server and to iron out how BerrySync is going to handle the RSA key pairs. I want to be able to talk to a production Firefox server during the coming week!

Friday, May 6, 2011

hello cdot! - BerrySync

My first week at CDOT, Seneca's Center for Developing Open Technology is coming to a close and there have been some wicked developments.

The Atmosphere
I'm really enjoying working here as a summer student. The orientation was welcoming, bundling in breakfast and lunch. That day all of the summer researchers were gathered together and given the history of CDOT as well as their expectations for the summer.

Respective project heads gave brief speeches about what their teams would be working on this summer, my project in particular ill mention farther down.

Every morning teams meet up for scrums where you briefly talk about what you didn't yesterday and your goals for today. I'm really enjoying these, it helps everyone keep in touch with the cool projects going on here and it also allows people to offer advice before it even hits IRC.

Speaking of IRC, we are all expected to be there. The name of the game is, be involved. Your knowledge can help people and the knowledge of your community is at your figure tips.

My Project
Creatively named BerrySync the short description is that, we aim to develop a enterprise product roughly equivalent to Firefox Home for the iPhone, except for the BlackBerry.We aren't promising a copy of the feature set nor are we limiting ourselves to Home's feature set, Home is a guideline. Similarly were investigating the interaction of FireFox Sync for integration and perhaps in the near future customization.

So this first week, I've been setting up my environment and figuring out what I need research in depth. I just got my computer and my dual boot is installing as we speak. I'll post again with updates on my current task / demo / I'm a learner by doing.