CFQueryReader Update: Support for ExtJS 5 and ColdFusion 11

Due to a donation from Bruce Lomasky (thanks Bruce), I've written a new CFQueryReader for ExtJS 5. CFQueryReader is an Open Source project, available on GitHub, that I began years ago to create a proper reader for parsing Adobe's native ColdFusion query JSON return format into something usable by ExtJS data stores. As ExtJS has changed over the years, I have updated the reader to accommodate new versions. But, I don't do much ExtJS development anymore. It's a great front-end app development framework, but I've changed jobs a few times since co-authoring the first books on the library, and just don't work with the technology on a day to day basis anymore. Bruce had a need, and offered to pay for the development of an updated reader. So I dove back in to the ExtJS pool, to help him fulfill his requirements and put a new reader out there.

ExtJS went through a major rewrite between versions 3 and 4, requiring an entirely new direction. While much is backward compatible, in the move from version 4 to version 5, there were some big changes to ExtJS' data packages. Sencha has always done a good job with creating a rich, dynamic framework for web application development. I have always been impressed with their commitment to improving the product, continuously modernizing the framework, and creating more of a "software craftsmanship" approach to web application development. That said they haven't always done such a great job with ensuring an easy upgrade path for developers. Changes to their data package completely broke some of the use cases of CFQueryReader, requiring some refactoring to accomadate.

And that's OK. Sencha's changes to their data packages are some welcome changes, especially for our little plugin. In all of the past revisions of CFQueryReader, we've extended a base Array Reader class, and written an override method of the core readRecords() method. While this worked, it was really kinda kludgy. What we really needed was a way to transform the incoming data packet prior to the reader really processing the object. With ExtJS, we now have that ability to do just that.

ExtJS 5 introduced a new configuration option for data readers: transform. This new configuration attribute takes a function within which you can manipulate the return data of a store's ajax request prior to the reader actually processing that data. This gives some underlying flexibility that wasn't really there before, especially when using Ext.Direct, but for now you really just need to know the basics.

ColdFusion upper cases object keys when it serializes to JSON. If you are manually creating the structure, there are ways to fix that:

view plain print about
1var myStruct = {
2 "key1": "some value,
3 "Key2": "this one is cased differently",
4 "differentKey3": "this one is more different"
5 };

This is fine, and helps a little bit, but ColdFusion creates an object when it serializes a ColdFusion query object, and you can't control it's casing:

view plain print about
1ColdFusionJson = {
2 COLUMNS: ["KEY1","KEY2","DIFFERENTKEY3"],
3 DATA: [
4 ["some value","this one is cased differently","this one is more different"]
5 ]
6 };

When you build your reader configurations, we no longer worry about this casing, as we will do a case insensitive matching of keys during the pre-process:

view plain print about
1reader: {
2 type: "cfquery",
3 rootProperty: "myQuery",
4 totalProperty: "totalCount"
5 }

NOTE: We will lowercase column names during preprocess, and you should remember that when creating your "dataIndex" attributes of your Model's Field configurations.

CFQueryReader will read ColdFusion query JSON serializations that are:

  1. The root of your return
  2. Nested within a larger return object (struct)
  3. Have been converted with ColdFusion QueryForGrid (not suggested, but supported)

And, because ColdFusion 11 includes the new "struct" type for the queryFormat parameter (even as an argument of your ajax request), CFQueryReader will properly parse that as well.

And, since we can now process the return prior to ExtJS calling it's internal readRecords() method, CFQueryReader no longer extends Ext.data.reader.ArrayReader as it's base class, but the more appropriate Ext.data.reader.JsonReader instead, allowing for a more native access approach under the hood.

I've made the new plugin available, for now, within the Ext5 branch of the repository. If you have the need, take it for a spin, and let me know if there's anything that might require tweaking.

ColdFusion + Ext JS Position: Los Angeles

This was forwarded to me yesterday. Anyone who might be interested, ping me on the contact form and I'll forward you the details:

ColdFusion Developer will be responsible for the development of financial reporting systems. He/She will also develop detailed system design and programming specifications, support daily business activity by providing technical support of systems and regularly meet with the IT project lead to review progress and problems related to the job. Additionally he/she will design intuitive and effective UIs using modern client-side technologies, including AJAX, Sencha Ext JS, and other JavaScript libraries.

ColdFusion developer with 5 years+ experience. Knowledge of ColdFusion CFC components, and MVC frameworks, and current feature set in ColdFusion 9.

Knowledge of (Sencha/4.1) Ext JS

Excellent knowledge of JavaScript, HTML, CSS, Web services, and importing Web-based data to Excel.

Good written and verbal communication skills.

Maintains a solid knowledge of information system software development life cycle and quality assurance

Demonstrates proficiency in systems analysis of business requirements.

Good knowledge of relational databases and SQL, experience with Oracle including writing stored procedures, triggers, views etc?

CFQueryReader v2.1: Now with metaData support

The Sencha guys just keep upping the bar, and Ext JS 4.1 is no exception. I've been reading Loiane Groner's Ext JS 4 First Look, to review it, and continually find new, cool stuff. 4.1, however, takes it even further. While upgrading CFQueryReader, I was working with 4.1 RC2. And, while extending the base classes, I came across a new feature that wasn't fully documented yet: adding metaData to a server-side store response for changing configuration on the fly. Sometimes it would be nice to just...change up. Now that 4.1 is fully released, I had to make sure that this worked in CFQueryReader.

So, last night I sat down and hammered out this functionality. It took a lot more than I realised, and I learned a lot more about the Ext JS internal code, but I think CFQueryReader is better for it. Consider the following ColdFusion method:

I reused my getEntries method, to get my query. Here, I'm creating a metaData object, at the root of the return, to define the dataset. Configuration options that I normally define client side (root, totalProperty, etc) I put in to the metaData key. When the response is received by the client, the reader will pass this metaData in to our app, applying this configuration to our reader, store, model, and so on. In the above method, we let the metaData map our columns to fields, rather than doing it client side (CFQueryReader will automatically skip the column mapping if metaData.fields is present in the response.) Our client side store might now look like this:

view plain print about
1Ext.create('Ext.data.Store', {
2    storeId: 'entryStore',
3    model: 'Entry',
4    remoteSort: true,
5    proxy: {
6        type: 'ajax',
7        url: '/com/cc/Blog/Entries.cfc',
8        extraParams: {
9            returnFormat: 'json',
10            method: 'getWithMeta'
11        },
12        limitParam: 'pageSize',
13        pageParam: 'pageIndex',
14        sortParam: 'sort',
15        reader: {
16            type: 'cfquery'
17        }
18    },
19    autoLoad: true
20});

A full example of this, in action, can be seen on a demo page of the CFQueryReader site. The full source code, of the example, can be found in the CFQueryReader GitHub repository.

Ext JS 4.1 Grid: Part1: Basic Config

Many moons ago, I wrote a series on My First Ext JS Data Grid. It was very popular. In fact, it got me the gig co-authoring two books on Ext JS and spawned an open source project targeted at integrating Ext JS with ColdFusion. But, I did that series back in 2007, using Ext JS 1.1 (maybe?), and an update is long overdue.

[More]

CFQueryReader 2.0: Site and Demo Updates

Note: I mistakenly posted this under the wrong title. I must get more sleep ;)

I finally got around to a major overhaul of the CFQueryReader site, including all new demos and documentation of the latest build for Ext JS 4.x.

The new demos for Ext JS 2.x and 3.x include links to legacy API's for both versions of the library. The 3.x version includes an example of using Ext Direct, and the new 4.x demo includes paging data grids.

(Big thanks to Loiane's Ext JS 4 First Look, which showed me very quickly how dead simple Ext JS grids and data stores have become.)

I have not yet tested CFQueryReader with Sencha Touch, but it should work. Any feedback just let me know.

Ext JS 4 and ColdFusion: CFQueryReader 2.0

Wow! It's been too long. I've been so buried playing with new stuff I haven't had much time to write about it. Time to rectify that.

My position over the last year and a half hasn't required me to use Ext JS. And without a project I really haven't had much time to dive in to Ext JS 4. But, I am reviewing Loiane's Ext JS 4 First Look right now, and decided the simplest way to get in was to apply what she was telling me. I knew some of the basics, but Ext JS 4 is a big change, and Loaine's book quickly helped me to grok differences in the class model, updates to the data api, and more. In no time at all I had completely rewritten CFQueryReader, for parsing the native JSON return of a ColdFusion query object. I need to verify, but I'm fairly certain there are more lines of comments than there are code, in the new file. And it is wicked simple. First, include the js file, then define a record of data:

[More]

The Joys of Developing for Internet Explorer

Note: Follow the madness here to it's conclusion, to discover yet another "Really? I didn't know that..." IE moment, that may save you heartache and pain.

Is the sarcasm evident in my title? It should be. While Internet Explorer may have been "groundbreaking" when it was released, it has ever been the bain of the web developer's existance. Why, you ask? Because it refuses to adhere to standards, and just operates differently than everyone else. To be fair, Internet Explorer 10 is in the works, and supposedly closes the gap a fair amount (and even wins out in some of the html 5/css 3 support), and 7, 8 and 9 did progressively improve (if slowly). Unfortunately the fact remains that some companies/organizations/governments are stuck on Internet Explorer 6, or maybe even 7. If you are a web developer, and write cross-browser web applications, chances are Internet Explorer has hit you more than once in your career.

It's the dumbest things that hit you too. The most obscure "WTF!?!" moments that drive you crazy. That is a daily experience for me now.

[More]

2011 In Review, and the View for 2012

My, how time flies when you're having fun! It seems like only yesterday that I was welcoming in 2011, and now we're here a year later. So many things have happened in the last year, and rereading that post I see that I missed some things I should've done, but let's take a look in retrospect.

I wrote 27 blog posts in 2011. This is nothing, compared to guys like Ray Camden or Ben Nadel, but for me it was quite a bit, especially when you consider that between March and August I released only one post. Very early in the year, I began a series on creatingmany sites with one codebase. In the process, the series has evolved to contain a fairly detailed primer in ColdFusion application architecture (because of it's importance to this process), has currently spanned 8 separate posts, and was even referenced by Sean Corfield in his great presentations on the same topic. 2012 will see the completion of that CF app discussion, and gradually move it back to the MSOC topic itself, as there is still a ton to talk about there, and a lot of interest in the topic. I also began a series on the jqGrid JQuery plugin. jqGrid is another Data Grid visualization tool (I have now written about three, including Ext JS and DataTables), and is a clear choice for those who must use JQuery. (To be fair, JQueryUI is working on a grid component, but they are still behind the curve, and way behind Sencha.) Finally, one common thread seen in the majority of my posts, is how much I've embraced cfscript. I wrote a lot of things, on a variety of topics, but most of my code examples were pure scripted examples.

Now let's talk about some other departures from the norm for Cutter.

You did not see a lot of content around Ext JS. In fact, I stopped writing Ext JS books. This is not, in any way, a reflection on my feelings for Ext JS. I still believe that Sencha has built one of the best client-side libraries for web application development. In evaluating the overall ROI, I realized that I was writing more for the community than the money, and that my reach was greater through my blog, while giving me flexibility on when and what I deliver from a content standpoint. That said, I didn't have a single project this year that used Ext JS, so had very little time to experiment and write about it. This year, I'm going to expand on a personal project, and get back to some great Ext JS content for my readers.

You, also, did not see me speak at any conferences this past year. Nor at any user group meetings. This wasn't because I didn't want to, but because of some more personal reasons. I'm not going to go in depth here, other than to say that I've had some long standing health issues that required me to have some surgery done on my mouth. (Mark Drew is making a joke right now...) Aside from the fact that this has been very costly (chewing up any conference/travel budget), it also meant that my speech has been affected for a good part of the year. Thankfully this experience is (mostly) over now, and I hope to get back to presenting sometime this year. Any user group looking for a speaker this year, please contact me through the Contact link on this blog.

One group I am hoping to speak to this year is the Northeast Florida CFUG. I have to call Mike back, but he's looking to get things kicked off again, and I want to help it be successful. If you're in or around the Jacksonville area, make sure to keep an eye on the site for upcoming events.

One other thing I'm looking to do is to migrate all of my projects into GitHub. I've been using Git at work, and I am loving it, and I think combining GitHub with RIAForge is a great way to promote the terrific technologies we work with every day. I will make the time, I promise.

This comes to the final discussion of this post, Adobe. I again had the pleasure of being an Adobe Community Professional this past year. Due to my health issues, I didn't get to do everything I would've wanted to this year, but I've tried to be a good supporter. There are some fabulous things coming in ColdFusion Zeus and, by extension, to ColdFusion Builder as well. There has been a lot of hub-bub over Adobe's communications flubs regarding Flash, mobile, and Flex. I've avoided much of the discussion, other than to say "be patient and watch". Flash isn't going away, and neither is Flex. HTML 5 is a beautiful thing, if you aren't developing desktop browser applications (i.e. You're only writing for mobile/tablet development). There, that is my whole contribution to that discussion. Give it a rest.

2012 will be a fantastic year. Set yourself some clear, definable goals. Break them down, step by step, and write the steps down on paper. Each successive step, print out in large letters and place it somewhere where you will see it each and every day. Set yourself up to succeed, and you will. Have a great year, everyone, and I can't wait to hear what you have planned for 2012.

How I Do Things: ColdFusion and Ajax Requests

I get a lot of questions about handling Ajax requests. What are some best practices? How do you format your requests? How do you taylor your functions in ColdFusion? Those sort of questions. We (the ColdFusion community) have embraced Ajax. Many of us spent so much time working mostly server side code on top of simple (and often poorly thought out and formatted) forms, but user habits have changed. And so have we. You have to move with the times to keep up in the digital rat race.

Let's start with the stuff most CFers already know; the server side stuff. When we make an Ajax request, most often we will hit a function within a CFC. We need data. You can hit a .cfm page, and return a bunch of preformatted HTML, but often that's a lot of data transfer when your really only need a few bits of data. HTML code can get heavy, what with all the tags and such. Similarly, you could return data in XML form (WDDX is native in ColdFusion). XML is sometimes a required format, for certain requests, but like HTML it can often be overly verbose. When working with server side requests from the client, you want to minimize data transfer as much as possible. That's why Flash performs best using the AMF format, and why most Ajax applications work with JSON.

When constructing your ColdFusion functions for remote requests, you don't have to write functions that can only be used via Ajax. In fact, that would probably be bad practice, as it would really minimize the code reusability of your functions. Basically, the only true requirement, at the server side, is that you set your function's access attribute to remote.

view plain print about
1/*    
2 *    COMPONENT SomeComponent
3 *    Just some example component
4 *
5 *    @name SomeComponent
6 *    @displayName SomeComponent
7 *    @output false
8 */

9component {
10    /*
11     *    FUNCTION SomeFunction
12     *    This is just some function you've written
13     *
14     *    @access remote
15     *    @returnType struct
16     *    @output false
17     */

18    function SomeFunction(required string arg1, required numeric arg2) {
19        var retVal = {"success" = true, "message" = "", "data" = ""};
20        // some function stuff here
21        return retVal
22    }
23}

That's it. Nothing to it. Did you see the required line? The one you need for making a remote request of this function?

view plain print about
1*    @access remote

Ok, we're off and running. Sort of. "Hey, Cutter, what's with returning a struct? I thought we were working with JSON?" Well, let me explain...

Have you ever worked on some Ajax call, loaded the page up in the browser, and nothing happened? You change variables, do this and that, and get nowhere? Finally you open up Firebug, watch the request, and see that you CF code has errored out for some reason. Instead of data coming back, the browser is getting the thousands of lines of HTML and Javascript that is the ColdFusion error display. In the immortal words of Charlie Brown: AAUUGGHH!

What happens if your user sees this behavior? Some variable isn't getting set when you think it is, or some db guy added or removed a column from your database without your knowledge. An error gets thrown, and you never even realize it. Ouch! Users are quick to abandon you. How can you gracefully handle these things?

Well, how do you deal with error handling within your application? If this answer is "I don't", then you might want to explore another career path. No, just kidding (kind of). If you don't use error handling, this is the perfect scenario to start. There is no reason why you're users should see (or, in this case, not see) these types of issues within your application. By carefully thinking things out, you can provide a better user experience.

view plain print about
1/*
2     *    FUNCTION SomeFunction
3     *    This is just some function you've written
4     *
5     *    @access remote
6     *    @returnType struct
7     *    @output false
8     */

9    function SomeFunction (boolean debug = false, required string arg1, required numeric arg2) {
10        var retVal = {"success" = true, "message" = "", "data" = ""};
11        var q = new Query();
12        try {
13            // do your query stuff, setting the result to the "data" key
14            if (!retVal["data"].recordCount) {
15                throw(type = "ourCustom", errorCode = "RC_000", message = "No records were returned matching your criteria.");
16            }
17        } catch (any excpt) {
18            LogTheError(excpt); // Custom error logging for us
19            retVal["success"] = false;
20            if (!ARGUMENTS.debug) {
21                retVal["data"] = ""; // Clear the "data" key, so we don't pass unneeded bits back to the client. Override this by passing 'debug' into the request.
22                if (excpt.type eq "ourCustom") {
23                    retVal["message"] = excpt.message;
24                    retVal["errorCode"] = excpt.errorCode;
25                } else {
26                    retVal["message"] = "There was an error in making your request, and our administrators have been notified.";
27                    retVal["errorCode"] = "UN_001"; // Internal error code for an undefined error
28                }
29            } else {
30                retVal["message"] = excpt.message;
31                // and anything else you might want
32            }
33        }
34        return retVal;
35    }

Now, there are probably better ways to write your catch statements, but I'm just trying to lay some foundation here. The basic idea is, taylor a message to send back to your users in the event of a failure. "ErrorCode?" Many enterprise products have custom error codes they use to denote that a specific error has occurred (like the above 'RC_000' for no returned records).

So, we have a success key, denoting whether the requested ColdFusion function did what we wanted. We have a data key that we use for returning the data generated by the request. And, we have a message key that we may, or may not, use only in the event of an error. We've also used and errorCode in this scenario, only in the event of an error. "Cutter, something doesn't look right?"

view plain print about
1// You use the quotes and this struct access notation to preserve the casing of key names, which is important in JavaScript
2    var retVal = {"success" = true, "message" = "", "data" = ""};
3    ...
4    retVal["message"] = excpt.message;

That should be the basics of the server side stuff. But, then the question comes, "What if our model isn't under the web root? What if the model CFC's aren't web accessible?" OK, fair enough. Then you'll need a proxy CFC. Let's regroup a sec, and show what this might look like.

view plain print about
1/*    
2 *    COMPONENT SomeComponent
3 *    Just some example component
4 *    This example sits outside the webroot, in a 'com' directory that's mapped in the CFIDE
5 *
6 *    @name SomeComponent
7 *    @displayName SomeComponent
8 *    @output false
9 */

10component {
11    /*
12    *    FUNCTION SomeFunction
13    *    This is just some function you've written
14    *
15    *    @access public
16    *    @returnType struct
17    *    @output false
18    */

19    function SomeFunction (boolean debug = false, required string arg1, required numeric arg2) {
20        var retVal = {"success" = true, "message" = "", "data" = ""};
21        var q = new Query();
22        try {
23            // do your query stuff, setting the result to the "data" key
24            
25            if (!retVal["data"].recordCount) {
26                throw(type = "ourCustom", errorCode = "RC_000", message = "No records were returned matching your criteria.");
27            }
28        } catch (any excpt) {
29            LogTheError(excpt); // Custom error logging for us
30            retVal["success"] = false;
31            if (!ARGUMENTS.debug) {
32                retVal["data"] = ""' // Clear the "data" key, so we don't pass unneeded bits back to the client. Override this by passing 'debug' into the request.
33                if (excpt.type eq "ourCustom") {
34                    retVal["message"] = excpt.message;
35                    retVal["errorCode"] = excpt.errorCode;
36                } else {
37                    retVal["message"] = "There was an error in making your request, and our administrators have been notified.";
38                    retVal["errorCode"] = "UN_001"; // Internal error code for an undefined error
39                }
40            } else {
41                retVal["message"] = excpt.message;
42                // and anything else you might want
43            }
44        }
45        return retVal;
46    }
47}

And the remoting proxy...

view plain print about
1/*    
2 *    COMPONENT SomeComponent_Remote
3 *    Just some example proxy component
4 *
5 *    @name SomeComponent_Remote
6 *    @displayName SomeComponent_Remote
7 *    @output false
8 *    @extends "com.cc.Examples.SomeComponent"
9 */

10component {
11    /*
12     *    FUNCTION SomeFunction
13     *    This is just some function you've written
14     *
15     *    @access remote
16     *    @returnType struct
17     *    @output false
18     */

19    function SomeFunction (boolean debug = false, required string arg1, required numeric arg2) {
20        return Super.SomeFunction(argumentCollection = ARGUMENTS);
21    }
22}

"Wait a second! I thought you said this post was about Ajax? Where's the JSON? Where's the Javascript?" Well, Ajax is useless without the data, right? Now that you have your CFC's and functions setup, let's look at the Javascript to make our requests. First we'll look at a basic JQuery Ajax call. First, let's set a global JS variable in the page.

view plain print about
1<cfscript>
2    param type = "boolean" name = "URL.debug" default = false;
3    
4    savecontent variable = "REQUEST.adder" {
5        WriteOutput('<script type="text/javascript">var debug = #URL.debug#;</script>');
6    }
7    REQUEST.addHeaderOutput &= REQUEST.adder;
8
</cfscript>
9...
10<cfhtmlhead text="#REQUEST.addHeaderOutput#" />

Next we'll setup our Ajax call.

view plain print about
1$.ajax({
2    url: '/my/cfc/location/SomeComponentRemote.cfc'
3    dataType: 'json',
4    data: {
5        method: 'SomeFunction', // The method we're calling
6        returnFormat: 'JSON', // Give us the return as JSON
7        debug: debug, // the global default 'debug' set earlier by ColdFusion
8        arg1: 'This is my argument',
9        arg2: 42
10    },
11    success: function (response, status, options) {
12        if (response.data) {
13            // Do something with the data
14        } else {
15            // This means the request failed, and response.data should contain
16            // a 'message' key, to present to the user, and an 'errorCode' key
17            // that you're application might act on
18        }
19    }
20});

Here's the same basic Ajax request, using Sencha's Ext JS.

view plain print about
1Ext.Ajax.request({
2    url: '/my/cfc/location/SomeComponentRemote.cfc',
3    params: {
4        method: 'SomeFunction', // The method we're calling
5        returnFormat: 'JSON', // Give us the return as JSON
6        debug: debug, // the global default 'debug' set earlier by ColdFusion
7        arg1: 'This is my argument',
8        arg2: 42
9    },
10    success: function (response, options) {
11        var response = Ext.util.JSON.decode(response.responseText)
12        if (response.data) {
13            // Do something with the data
14        } else {
15            // This means the request failed, and response.data should contain
16            // a 'message' key, to present to the user, and an 'errorCode' key
17            // that you're application might act on
18        }
19    }
20});

I'm always surprised at how few CFers know this little bit. In case you missed it, the key piece to your request parameters comes down to one important argument.

view plain print about
1returnType: 'JSON'

When this argument is passed in the Ajax request parameters, ColdFusion will automatically serialize your native ColdFusion response objects into JSON. This is what allows you to set your returnType to a struct in your function definitions.

Most of this stuff has been around for quite a while. Some use it a lot more than others. Some are just afraid of client side stuff in general. There's a ton of material out there on how to do these things. This is my way of working with it. That doesn't make it right, or better than anybody else's. It's just what I've developed as a working pattern over the years. How do you handle it?

The Next Wave: Ext JS 4 Beta 1

I only have a few minutes, as I am totally buried right now, but I wanted to reach out to everybody to spread the word about Ext JS 4 Beta 1 being released yesterday. Some of you may have been following the Developer Previews on the Sencha Blog, and if you have you already know some of the amazing work coming out of Sencha for this release. If you haven't, well let me give you a really quick recap of some of what you could expect:

There's also an entirely new rendering engine under the hood, new components, and more. You can even run Ext JS 3 & 4 on the same page, if needed. Go to the Sencha site to see some great examples of what you can do. It is still a beta, so the forums are pretty busy with people learning the ropes and filing bugs as they're found, but the foundation has been laid for a truly ground-breaking update to this library.

More Entries