CFQueryReader Update: Support for ExtJS 5 and ColdFusion 11

Due to a donation from Bruce Lomasky (thanks Bruce), I've written a new CFQueryReader for ExtJS 5. CFQueryReader is an Open Source project, available on GitHub, that I began years ago to create a proper reader for parsing Adobe's native ColdFusion query JSON return format into something usable by ExtJS data stores. As ExtJS has changed over the years, I have updated the reader to accommodate new versions. But, I don't do much ExtJS development anymore. It's a great front-end app development framework, but I've changed jobs a few times since co-authoring the first books on the library, and just don't work with the technology on a day to day basis anymore. Bruce had a need, and offered to pay for the development of an updated reader. So I dove back in to the ExtJS pool, to help him fulfill his requirements and put a new reader out there.

ExtJS went through a major rewrite between versions 3 and 4, requiring an entirely new direction. While much is backward compatible, in the move from version 4 to version 5, there were some big changes to ExtJS' data packages. Sencha has always done a good job with creating a rich, dynamic framework for web application development. I have always been impressed with their commitment to improving the product, continuously modernizing the framework, and creating more of a "software craftsmanship" approach to web application development. That said they haven't always done such a great job with ensuring an easy upgrade path for developers. Changes to their data package completely broke some of the use cases of CFQueryReader, requiring some refactoring to accomadate.

And that's OK. Sencha's changes to their data packages are some welcome changes, especially for our little plugin. In all of the past revisions of CFQueryReader, we've extended a base Array Reader class, and written an override method of the core readRecords() method. While this worked, it was really kinda kludgy. What we really needed was a way to transform the incoming data packet prior to the reader really processing the object. With ExtJS, we now have that ability to do just that.

ExtJS 5 introduced a new configuration option for data readers: transform. This new configuration attribute takes a function within which you can manipulate the return data of a store's ajax request prior to the reader actually processing that data. This gives some underlying flexibility that wasn't really there before, especially when using Ext.Direct, but for now you really just need to know the basics.

ColdFusion upper cases object keys when it serializes to JSON. If you are manually creating the structure, there are ways to fix that:

view plain print about
1var myStruct = {
2 "key1": "some value,
3 "Key2": "this one is cased differently",
4 "differentKey3": "this one is more different"
5 };

This is fine, and helps a little bit, but ColdFusion creates an object when it serializes a ColdFusion query object, and you can't control it's casing:

view plain print about
1ColdFusionJson = {
3 DATA: [
4 ["some value","this one is cased differently","this one is more different"]
5 ]
6 };

When you build your reader configurations, we no longer worry about this casing, as we will do a case insensitive matching of keys during the pre-process:

view plain print about
1reader: {
2 type: "cfquery",
3 rootProperty: "myQuery",
4 totalProperty: "totalCount"
5 }

NOTE: We will lowercase column names during preprocess, and you should remember that when creating your "dataIndex" attributes of your Model's Field configurations.

CFQueryReader will read ColdFusion query JSON serializations that are:

  1. The root of your return
  2. Nested within a larger return object (struct)
  3. Have been converted with ColdFusion QueryForGrid (not suggested, but supported)

And, because ColdFusion 11 includes the new "struct" type for the queryFormat parameter (even as an argument of your ajax request), CFQueryReader will properly parse that as well.

And, since we can now process the return prior to ExtJS calling it's internal readRecords() method, CFQueryReader no longer extends as it's base class, but the more appropriate instead, allowing for a more native access approach under the hood.

I've made the new plugin available, for now, within the Ext5 branch of the repository. If you have the need, take it for a spin, and let me know if there's anything that might require tweaking.

What's Wrong With ColdFusion - 2014 Edition

Well, it's that time again. Later this week is the second annual ColdFusion Summit, put on by Adobe. Last year, just prior to the summit, I posed the question What's Wrong With ColdFusion. The feedback was immediate, and intense, with many people weighing in with their thoughts on what could be done to progress ColdFusion, both as a platform and a language.

Unfortunately, last year I wasn't able to attend the summit, and development of the ColdFusion 11 server was nearly complete, so most of that feedback had zero impact on the new release. While their were many advancements in security and JSON processing, some of the other introductions were seemingly underwhelming, or even unwanted, to many long time CF developers.

Now, to be fair to Adobe, and the ColdFusion product development team, they do have a responsibility to cater to their paying customers. Developers are rarely the ones buying the server itself, and the money handlers buying the server licenses rarely have enough real understanding of the development process to truly provide reasonable feedback when asked "What do we do next?"

Past issues aside, the ColdFusion product team is beginning to explore what they will do with the next version of the ColdFusion server, so now is a good time to begin the discussion again as to what Adobe might do the next time around. It is a delicate balance, so our feedback to the team needs to be positive, constructive, and backed by hard arguments to support our requests.

ColdFusion has been known, since the very beginning, for making the hard things easy, with it's first inroads being in the are of data connectivity. ColdFusion makes it extremely easy to connect with a wide range of datasources. This was probably the first true task of CFML (originally called DBML). It's meeting this type of root level need that ColdFusion might need to focus on going forward.

There were a number of core concepts that came out of last year's discussion:

  • Script Language Overhaul
  • Command Line Interface
  • Package Manager
  • Modular Architecture
  • Support for NoSQL Databases
  • Application Profiling

There are some great suggestions here, and Rupesh actually had included these in his final slide of the cf.Objective keynote presentations as possible items of inclusion in the next version of the ColdFusion server. I, personally, say to Adobe that it is important to review the past year to help define the focus for the future. It is important for the product team to not waste time on solutions that are already available, or that developers can already do themselves with a little study. In that vein, the CLI and Package Manager may not be areas for Adobe to spend time on, since we now have CommandBox available to us. Focusing on solutions that are already freely available to us seems to be a waste of time and resources, unless you intend to go above and beyond any available solution. A key question to ask, in this type of evaluation, is not only "is this already available?" but also "does this truly fit the purpose, enhancement, and growth of our product?"

Modular Architecture

So, what sort of things can Adobe do? Well, the Modular Architecture concept has multiple levels of benefit, not only for developers but for Adobe as well. By creating a "core" build of ColdFusion (db interactivity, includes, custom tags and the like), with "packages" of additional functionality (ORM, CFFORM, CFCLIENT, etc) that could be installed only when required, Abode creates an architecture where developers can customize their environment to the needs of their application. This decreases the overall footprint of the server, removes the "cruft" that might be unnecessary, and expands the licensing options for Adobe as well. I also believe this could significantly improve the cloud options that could be offered for the server as well, which could greatly improve adoption of ColdFusion as a platform.

Image Processing

Image manipulation was a long time coming to ColdFusion, and was a great addition once it finally arrived natively. That said, the current implementation is very process intensive and slow, and support isn't quite what you would expect from a brand like Adobe. It may be time to revisit image processing in ColdFusion, possibly even exploring avenues other than Java for handling these processes, to improve quality and performance.

Server Performance

There is strong evidence pointing to other CFML engines (Railo) as having much better overall server performance. It is difficult to allocate time and resources to something that currently works. But, just because it functions doesn't mean it shouldn't be improved upon. Response time is a critical piece to any application, and Adobe ColdFusion (as a paid and licensed product) should be the best of breed in this category.

Language Overhaul

Again, it's difficult to put time and effort into something that already works. But, the current syntax of script and functions within ColdFusion is a barrier to adoption of the server. This may be one of the biggest, and most asked for, enhancements that Adobe could provide to ColdFusion. Developers, who have traditionally worked in other languages, spend little time evaluating ColdFusion as an option because of the belief that ColdFusion is a second class language. Why would you use ArrayAppend() when every other language uses array.push()? Many objects do provide Java level object access, but many of these are undocumented, or aren't displayed in the documentation as the first line of how something should be done. While ColdFusion must support the current language constructs for some time to come, it really is time to make a more ECMAScript compliant form of script for ColdFusion. Or, at minimum, something much more uniform and familiar.

What Next?

So, with the summit starting later this week, I pose the question again "How can ColdFusion be better?" What would you like to see Adobe focus on for the next version of the ColdFusion server? And why? How? To what purpose? What do you think are areas where Adobe should absolutely steer clear of? It is important that we provide our feedback now, early in this process, and that we do so in a positive and constructive manor.

I will bring any comments given here back to the ColdFusion product team at the summit later this week (yes, I'm presenting this year). I encourage everyone who is going to the summit to voice your opinions directly to the product team as well. I look forward to hearing everyone else's input on this.

Mailgun and ColdFusion

A while back I was playing around with Ghost, a blogging platform written for NodeJS. Crazy simple to setup, one of the nuggets that I found during the 5 min install was a reference to Mailgun.

Mailgun is advertised as "The Email Service For Developers", allowing you to send 10,000 emails for free every month. It has a nice API, and is very simple to use. A common problem that many ColdFusion developers have, when writing an app, is having access to an SMTP server solely for development practices. You can setup a local server, use some Windows SMTP service, or traffic mail through your personal accounts, but it really is better to have a sandbox that's a bit more separated, for security, sanity, and other reasons. ColdFusion 10+ have a built in SpoolMail utility (based loosely on Ray's project), but sometimes you just need to see the real output in a mail client instead of the browser.

Setting up an account was easy on Mailgun. Right from the homepage was a link to the free sign up page. Once the account was created I was given a Mailgun Subdomain. This is my sandbox to play in, so I clicked on the domain name, for the domain admin page, then clicked on the "Manage SMTP Credentials" link on the "Default SMTP Login" line item. From this page I was able to create a new sender (you can use the default if you wish, but sometimes I'll setup separate addresses for specific tracking purposes).

The final stage is to put the settings in the ColdFusion Administrator. From the mail settings area I entered my new "sender" address as my SMTP server username, the password that I created with the account, set the SMTP server address to "", and changed the SMTP port to 587. (The default port 25 will NOT work, but 587 with no other settings changes will work fine.)

Now I can send mail with ColdFusion from my dev environment without having to load my own SMTP server or subbing out my personal addresses.

ColdFusion UI: cfwindow

Not sure why I forgot to blog this, but a few weeks back Ray Camden told everyone that he wasn't going to tell you to stop using ColdFusion UI tags anymore. Instead, he and Adam Cameron started the ColdFusion UI the Right Way project. This project is a collection of articles and demo assets to show people that you can use other methods for displaying these dynamic client-side "widgets" in your applications, without using the CFUI tags.

Why? Well, it was a nice concept, but honestly (and in my continuing opinion) ColdFusion doesn't need to do UI. Yes, cfoutput, and looping over queries is fine, but tags like <cfform>, <cfpod>, and <cfwindow> are the devil's work. There have been long running issues with letting ColdFusion write your HTML, CSS and JavaScript for you, the chief among them being that they're bloated, buggy, limited, and (ultimately) unnecessary. You have a much finer degree of control over your application, and how it functions, when you write it yourself.

The CFUI tags were provided to allow the lowest common denominator (i.e. someone who can't really write code) to crank out something that works. They are not intended for skilled developers who write web sites and applications for a living.

And so, Rey kicked off the project with an article on replacing <cftootip> with the jQueryUI tooltip plugin, and I wrote the next article on replacing <cfwindow> with Bootstrap's modal plugin. I just checked the repository, and now there are also articles for <cfajaxproxy> and <cfdiv>.

The idea of the project is to show you a) that it's possible to do these things without the CFUI tags, and b) show you examples using a variety of libraries, to reiterate that you aren't limited to a single option.

Build A Better ColdFusion App: Simple Conditionals

In really large apps, every line of processing can become critical. Every small process can add up, in terms of overall overhead. Especially in high traffic apps. Sometimes, the little things can really make a difference. Even your simple code might be...simpler. Take the following example:

view plain print about
1<cfset mySelections = "1">
2<cfif #mine# eq "0">
3    <cfset mySelections = "0">

I saw something like this in code this morning. Right off, the first thing you see is that all of the variables are unscoped, forcing the server to do a scopeCheck() to find out which scope each variable is a part of. Then there's the unnecessary bits to go along with it, like setting a variable, then checking another to reset the variable based on a condition. There's also the bit where the one variable is unnecessarily hashed. On top of all of that, variable assignments are one of those things it just makes more sense to write in CFScript. Let's take a look at how this might be improved:

view plain print about
1REQUEST.mySelections = (URL.mine) ? 1 : 0;

One line of code. Less keystrokes. Clearer references, and logic.

The "mySelections" variable, in this instance, is a part of the REQUEST scope (I upper case all variable scopes, to make them easier to spot when editing code).

In this situation, "mine" was a URL variable, param'd further up the page. We've used short-circuit boolean logic in our conditional statement. For those that don't understand that, any numeric value other than "0" is evaluated to true. If the value is "0", then it is false.

And speaking of numeric values, you don't need to quote them. Even though ColdFusion would figure it out, a quoted value is a string. Don't quote numeric, and don't quote Booleans.

We use the URL conditional inside of a ternary operator. Ternary operators were introduced in ColdFusion 9. The basics of this are, if the conditional is true, then use the value after the question mark. If the conditional is false, then use the value after the colon.

Finally, there's no need to hash the URL variable. You only need to hash variable references as part of output, or when they are used inside of quotation marks in code.

Now comes the really fun part. There really isn't any need for REQUEST.mySelections at all. Since the value or URL.mine was param'd, further up in the code, then the variable is always available here. Rather than copying that value to another variable (which takes up more memory), we can just reference URL.mine anywhere that REQUEST.mySelections was being used.

As you maintain your applications, it's always good to take some time and read through what's been written before. Refactoring little nuggets like this one, and a little testing, can eventually go a long way in preserving the life of your app and your server.

Out with 2013, and in with 2014

2013 was Exciting! I had several side contracts, which helped us buy a house. We left Jacksonville, and moved back to the Nashville area. I finally put some health issues behind me, allowing me to get out to some user group meetings again, as well as present. I put in a new topic for cf.Objective, which has been accepted. I picked up the camera I've been waiting a decade to buy, and jumped back in to photography with both feet. And, I carved in some time there to review a few books, do a CF Hour interview, and write a few blog posts (some even a little controversial). Now, there's downside here too. Heavy side contract work is not always conducive to a happy home. Buying a house, much less from another state (and the move that goes along with it) can be very grueling and stressful. Working at this kind of pace can bring on other health issues. The financial requirements of buying a house, moving state to state, setting up a new home, etc, make it impossible to do "other things" (like conferences, for instance). And, I still have several other book reviews I've yet to write, and didn't blog nearly as much as I would've liked. Well, you do what you can with what you have, and pray for the best. The two bright points in all of that are my wife, Teresa, and my daughter, Savannah, who continually support this crazed life, and without whom it really wouldn't be worth doing it all. I can never tell them enough how much I Love Them, nor just how much they mean to me. I never got the opportunity to talk about my conversation with Rakshith, concerning the future of ColdFusion. It was a great conversation, focusing quite a bit on Adobe's commitment to the platform, the strides in the education initiatives, and a focus on giving developers the tools (and language) they need to build better products. I think one of the greatest ColdFusion bits, to come out of this past year, was the creation of Team CF Advance. This group, entirely made up of volunteers, is working to produce Open Source applications and API's in ColdFusion. While it is still getting off the ground, the team has already made some pretty big strides. This is the kind of thing that will help bring ColdFusion back out front. I have high hopes, and high expectations, for what is to come from 2014. I've already committed to taking at least one picture a day, gotten seriously started on losing a few (too many) pounds, have several topics laid out for blog series, and am about to put some spit and polish on that preso I mentioned earlier. And that should be just the beginning. So, to all of my friends, colleagues, co-workers, and family, I wish you all the Happiest and most Prosperous of New Years.

What I'd Like To See Come Out of the ColdFusion Summit

So, the very first Adobe ColdFusion Summit is being held this week in Las Vegas, NV. What started out as a small conference, originally capped to 250 participants, is now nearly twice that size, with people coming from all over the world to attend.

We've had a lot of discussions here recently about What's Wrong With ColdFusion, and things that can be done to fix those perceptions. Ultimately, there's been a lot of positive feedback, some reasonable initial response out of Adobe, and a community that's started talking again.

What I'm hoping for is for that discussion to continue, hard, at the Summit. As much as I would like to be there, due to constraints I am unable to attend. So, I'm hoping that the community will come together to ask some hard questions. What really is being done to address the concerns brought up? How does this change current development practice, timeline, and direction? How soon can we expect to see x, y, or z? When, and how, is marketing of ColdFusion going to change?

It's important that we, the community, use this opportunity to speak to some of the engineers. "Why are we doing this? (in this new version) And not doing that?" We have to tell them what we think, and why, to assist in defining new directions. They're always gathering requirements, and you are the perfect focus group. The suits (management types that buy the licenses) can tell them that it should do this or that, but you (the developer) are the ones who truly know the things that are needed.

I know that I've talked a lot about "Adobe" ColdFusion. I am an Adobe Community Professional, so I have to choose my words carefully sometimes. Ultimately, any positive changes are changes that push innovation, competition, and advancement of the platform. In that vain, the Team CF Advance initiative, to write, support, and contribute to Open Source software solutions written with ColdFusion, is one of the single greatest things to come out of our discussion. (Big kudos to Denny Springle [with a side of Corfield], for pulling this idea back out of the hat and getting it up and running) Any time, and assistance, that anyone can give, is a good thing. While we need people to write code, we also need people that can assist with requirements gathering, documentation, and other things. Of all of the suggestions pushed in these discussions, this is the one that we (the developers) can have the most say and control.

Why I Like, and Use, CFScript

Many, many moons ago, I began a journey to learn web development. Actually it was a re-acquaintance, as I had gotten side tracked for a few years in a poor business partnership, having to catch up on how much things had changed. During that time, HTML had moved for v2 to 4, CSS was the crazy new thing replacing font and center tags, Dynamic HTML was making JavaScript hot again, and the browser wars were making life impossible.

Back then I was on the LA Tech Edu JavaScript mailing list (sadly retired, years ago). This guy named Peter Paul Koch was a major contributor to the list, and just ramping up this little site called QuirksMode. Scripting, though somewhat nightmarish with the browser wars, was actually a lot of fun. My first shopping cart was completely JS based.

I have always pushed for full parity between cfscript and tags, because scripting is more natural to me, in terms of writing programmatic logic. Sure, cfoutput and cfloop make a lot of sense in the middle of html display code, but CFC's don't (typically) contain display code, being data access and model objects.

I have issues with the implementation of CFScript. I find that there are a lot of functions of similar actions with completely different naming conventions, and argument requirements, and more. I think these things need to be addressed. That said, if I have to call the function anyway, I prefer script to tags when it's outside of display. It's a more natural approach, to me. 99 times out of 100 I end up with less lines of code, and find the logic more defined, clear cut, and easier to understand.

It's not for everyone. There are many that just plain hate cfscript, and that's their prerogative. But I will continue to use cfscript, and my examples will show that, because I personally find it a better way to write code. If it's not your preference, that's fine, you're welcome to translate my examples to tags if you need to, but for me this is how I do it. (For the record, many of the newer functions are not identical, under the hood [base Java code], as their tag counterparts, and some scripted functions have been proven to have better performance than their tag counterparts. There are benchmarks for this out there, somewhere, if you feel strongly enough to go look for them. I just take it on my experience with using it.)

Build A Better ColdFusion App: When The Var Doesn't Exist

Author's Note: Due to some of the feedback below, I'm going to remove a bit of example regarding paraming objects (see the feedback). Some have actually benchmarked where this creates overhead, rather than improvement of performance, so I'm going to stick with simple variables as this is researched further.

So, let's kick off this round of "Build a Better CF App" with a simple bit of code that you might see quite often in an app:

view plain print about
1<cfif NOT StructKeyExists(FORM, "isactive")>
2    <cfset FORM.isactive = false>

Simple, right? Probably dozens of these hanging around in your code. Now, let's look at a simpler form, that also gives us the added benefit of type checking.

view plain print about
2    param name="FORM.isactive" type="boolean" default=false;

What? Huh? Yeah, this makes sense. See, a checkbox type formfield doesn't actually get passed on form submission if it isn't checked, so chances are you're running this same code, in one way or the other, to default that value for you. Utilizing the param here (cfparam in tag form, but who's using tags outside of display stuff anymore? [hint, hint]) is really the better way of handling this.

  1. You get a default value for the variable
  2. The default is overridden, if the variable actually occurs
  3. If the variable does exist, the param ensures that it is of the proper type
  4. But wait, let's throw in another similar bit of code for good measure. How many times have you seen something like this?:

    view plain print about
    1<cffunction name="getUser" output="false" returntype="User">
    2    <cfargument name="firstName" type="string" required="true">
    3    <cfargument name="lastName" type="string" required="true">
    5    <cfif NOT StructKeyExists(ARGUMENTS, "userID")>
    6        <cfset ARGUMENTS.userID = 0>
    7    </cfif>
    9    <!--- More code --->
    10    <cfreturn LOCAL.User>

    Just doesn't feel right, does it? Especially when ColdFusion already gives us a way to handle this scenario:

    view plain print about
    1public com.mysite.User function getUser (required string firstName, required string lastName, numeric userID=0) {
    2    // More code
    3    return LOCAL.User;

    You can apply a default value to any argument of a function, utilizing your arguments just as you would a param.

    Now, there may be some valid reason for not paraming a variable, and using conditionals in some way similar to those above. They're few and far between, but there are exceptions to most every rule, so you'll need to evaluate that when the time comes.

New Series: Build A Better ColdFusion App

One common complaint, among opponents of ColdFusion, is the poor code that's out there on the web. I would counter that there's a ton of bad code out there in most any language (I know, I've written some of it), but I do "get it". ColdFusion, as powerful a language as it can be, also has an extremely low barrier of entry. It was created in such a way that anyone who could write HTML could turn around and write an application in ColdFusion (hence the language design decision to use a tag syntax).

Well, just because you can write code in one way, that doesn't necessarily mean that you should. There are thousands upon thousands of blog posts, articles and books out there showing people how to do various things in ColdFusion. Many (not all) are cobbled together rather quickly, offering up functional examples on how to do this or that. And, while awesome that someone bothered to take that time and share that knowledge, they didn't always consider best practices in their examples. After all, it was just a quick example, right? You wouldn't just copy and paste it into your editor. You'd tailor it to your application's needs, right?

Well, no. Many of us have copied some bit of code and just dropped it in without revision. And often it just stays like that, until it causes some sort of issue. To top it all off, chances are you copied and pasted the same bits of code into other areas of your app, or even into new applications, again without revision.

So, I want to start a new series here at Cutter's Crossing. I'm going to call it "Build A Better ColdFusion App", and in each article I want to highlight some small bit of best practice advice, with some code examples that match up accordingly. It won't be a "Tip of the Day" bit or anything, and the articles may come at any time, but they'll all get a common header, and I'll try to group them together so they're easy to find. If you have suggestions for some of these tips, or topic areas you think would be a good fit, please fill out the Contact Form and drop me a note with your suggestions.

By The Way: I'm not perfect. I might put up something and it's hosed too, or you might have suggestions on how to do it differently and/or better. Please, call me on it. We're a community of professionals, and want to put out info of value for any and all.

More Entries