Monday, February 7, 2011

(Part 3) AppEngine Task Queue + Transactions behavior

In part 2, I had shown a demo app that isolated some “bad” behavior with respect to using a transaction for persisting data to the data store and queuing a task to act on that data. One of my conclusions at the end of that exercise was to scrutinize JDO in another test. I have now done this.

I created a third test case where I access the raw datastore without using JDO. My initial results show that using the raw datastore API for transactions and queuing tasks works as you would expect. The queued tasks are advancing the state correctly without getting “old” state values.


To run this test, open the feedback channel, then  press the “Start Test 3 (sans JDO)” button. You should see the correct state arrival without mismatches. However, I experienced “retries” because of contention on the datastore, but it behaves correctly. This implies how much quicker the raw datastore API is too. Be sure to get the final results by pressing the “Get Test Status” button. The client adds invocations as they arrive, which can have more messages arrive than actually get atomically counted on the server.  (You can accurately measure everything by analyzing log files. But trying to tally things nicely for a demo GUI is a bear!)

Also, while writing these “low level” versions of the code, I discovered that the JDO transactions I had written previously actually allow you to do another thing you shouldn’t, and quietly lets you shoot yourself in the foot! I had a JDO transaction that initialized two objects with zeros and persisted them both to the DataStore. But they did not have a common parent! With the raw API, you get slapped on the wrist at runtime telling you that the two objects must belong to the same parent to live in the same transaction. Duh – I know this from the docs, but yet that little thing slipped by while writing the JDO version of the code.

This seems broken to me. I should get a runtime error in the JDO version of the code too.

Hmmm, I think I need to check out Slim3…