Ext JS 4 and ColdFusion: CFQueryReader 2.0

Wow! It's been too long. I've been so buried playing with new stuff I haven't had much time to write about it. Time to rectify that.

My position over the last year and a half hasn't required me to use Ext JS. And without a project I really haven't had much time to dive in to Ext JS 4. But, I am reviewing Loiane's Ext JS 4 First Look right now, and decided the simplest way to get in was to apply what she was telling me. I knew some of the basics, but Ext JS 4 is a big change, and Loaine's book quickly helped me to grok differences in the class model, updates to the data api, and more. In no time at all I had completely rewritten CFQueryReader, for parsing the native JSON return of a ColdFusion query object. I need to verify, but I'm fairly certain there are more lines of comments than there are code, in the new file. And it is wicked simple. First, include the js file, then define a record of data:


Intro to jqGrid Part 7: Grouping

OK, I know I said I was done, but I kept playing around and figured I should share. jqGrid, like other (read: Ext JS) good grid implementations, has the ability to group your data. What does that mean exactly? Well, let's look at our demo for inspiration. We've created a grid of blog post entries. Entries have associated Categories. We can reconfigure our grid to show the entries grouped by Category, with Category headers and accordion like separation.


ColdFusion Security Hotfix and Big Forms

The other day, Adobe released a new Security Hotfix for it's ColdFusion server. There were a number of things addressed in the hotfix, to help protect against Denial of Service attack using a hash algorithm collision. (My wife would say I sound like Charlie Brown's teacher right about now.) Ok, the important thing is you need to update your server.

Now for the fun part. We loaded the fix to our testing servers to run our app around the block prior to pushing this up to production. And, it's a good thing we did. We're preparing for a large deployment, and testing is pretty heavy right now. First thing in is that a form would no longer submit, throwing a 500 error every time. I didn't show me a 500 error, just a blank page. I had to look at Firebug to see the error code response. Now, if you've ever encountered a 500 error from the server then you know they don't typically tell you much. I reproduced the error locally and then went looking through the log files on the server.

In a multi-server configuration there are two core areas to look at log files. The first are the basic JRun logs. On a Windows systems, these files are located in the C:\JRun4\logs folder. Here you will typically find a {instance}-out.log file, and a {instance}-event.log file, for each ColdFusion instance you have configured. Right out of the gate you have admin-event and admin-out logs for the JRun administrator, and cfusion-event and cfusion-out for the default ColdFusion instance. I checked both files for my instance, and saw there weren't any items to tell me about the 500 error, so I then went looking at the ColdFusion logs.

Each ColdFusion instance has it's own set of log files, that you can see in the logs viewer in the ColdFusion Administrator. That said, the ColdFusion Administrator is not really the best place to go through these files, especially when you're really having issues. At this point, you just want to open them yourself. First, you have to find them. You do this through your instance, C:\JRun4\servers\{instance}\cfusion.ear\cfusion.war\WEB-INF\cfusion\logs. You'll probably find multiple log files here, from the application and eventgateway and mail logs, to individual logs from cflog calls.

Our issue, with submitting our form, was answered by the JRun -event logs, which gave me a few error messages saying something like this:

view plain print about
103/27 07:52:00 error ROOT CAUSE:
2coldfusion.filter.FormScope$PostParametersLimitExceededException: POST parameters exceeds the maximum limit specified in the server.
3    at coldfusion.filter.FormScope.parseQueryString(FormScope.java:397)
4    at coldfusion.filter.FormScope.parsePostData(FormScope.java:346)
5    at coldfusion.filter.FormScope.fillForm(FormScope.java:296)
6    at coldfusion.filter.FusionContext.SymTab_initForRequest(FusionContext.java:377)
7    at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:33)
8    at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
9    at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)
10    at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:126)
11    at coldfusion.CfmServlet.service(CfmServlet.java:200)
12    at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
13    at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
14    at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
15    at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
16    at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
17    at jrun.servlet.FilterChain.service(FilterChain.java:101)
18    at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
19    at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
20    at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
21    at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
22    at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
23    at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
24    at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

"Dude! What is that!?!" Well, luckily I had installed my security hotfix, locally, just that morning, so I remember reading over the instructions. In it's notes it had stated the following:

  1. Customers who want to change postParameterLimit, go to {ColdFusion-Home}/lib for Server installation or {ColdFusion-Home}/WEB-INF/cfusion/lib for Multiserver or J2EE installation. Open file neo-runtime.xml, after line
    view plain print about
    1"<var name='postSizeLimit'><number>100.0</number></var>"
    add the below line and you can change 100 with desired number.
    view plain print about
    1"<var name='postParametersLimit'><number>100.0</number></var>"

Just a heads up, that neo-runtime.xml file is minified, so you'll want to Find "postSizeLimit" to get that statement in the right place. We tried that postParametersLimit value (100) and found that our form had more than that (many were hidden, but that's another post all together), so we adjusted the number to 200. After restarting the instance again, we tested the form once more with complete success.

Hopefully this will help someone else avoid this issue. It's important to remember that Adobe does try to document these types of situations with hotfixes, so when you run into issues they should be your first source of troubleshooting information.

Intro to jqGrid Part 6: The Action Column

So far we've covered the base grid configuration, populating the grid with remote data, controlling layout by refining the column model, and given an example of event handling by binding a search form to our grid. This covers most of what the average developer might need to know about working with jqGrid, but I've been promising to tell you how to work with the icons we setup in our action column.


ColdFusion JSON Serialization Changes

Because I've had to search on this three times in the last year, I thought it'd be a good idea to document this here, for myself and others. There were a number of important changes that can trip you up from version to version and, if you know which version the changes came in it can be beneficial.

First is a change that affects JSON output, though it's not specifically about JSON. The ColdFusion 9.01 updater (the current base install, at the time of this post) addressed Bug 82980 by removing ColdFusion Debug Output from direct CFC requests. This was only important if you had debugging output turned on in the Administrator, but caused lots of issues when testing ajax applications in development environments that kept that setting on by default.

Unfortunately, Adobe made some changes to JSON serialization that were not only unpopular, but technically dead wrong. They were trying to resolve the treatment of numbers in JSON serialization. Numbers were being converted to floats (1 became 1.0, 12 became 12.0, and so forth), so Adobe changed that by converting all numbers into strings (11 became "11", 17.24 became "17.24"). This was wrong too, as numbers weren't numeric anymore. At the time it raised quite a stir.

They fixed this (Bug 83638) really quickly, releasingCumulative Hotfix 1 for ColdFusion 9.01, which fixed the issue of integers being converted to float and having them as numeric values (ie: [1,12,true,"this var",17.24]). Hurray! Except....

Enter the improperly titled Cumulative Hotfix 2 for ColdFusion 9.01. I say this because it really isn't 'cumulative'. Most things are straight, from a cumulative standpoint, with the seeming exception of the fix to JSON formatting. If you have not applied CHF 1 prior to installing CHF 2, then you will still have issues with JSON formatting.

Overall, Hotfix 2 is great, fixing many bugs. The install is crazy/scary, and must be followed to the letter to prevent major issues, but it's worth it. As long as you did apply CHF 1 first, that is. Thankfully the hotfix/upgrade install process is much better in ColdFusion 10.

Big thanks to Ray for helping me to realize that I wasn't crazy, and pointing me in the right directions for all of this info. Hopefully this helps someone down the line.

ColdFusion 10 Public Beta

Yes, everyone is talking about it. That's why I waited a day or two to post the info. Adobe has put ColdFusion 10 up on Adobe Labs, opening it's public beta cycle prior to full release. That's right folks, ColdFusion is finally hitting the double digits, and this release just continues to improve this great platform.

There are some big changes. First are some overdue upgrades, like retiring the JRun JEE server, replacing it with Tomcat. Web Services support is upgraded, moving to more current form of Axis, and adding native support for creating RESTful web services. Exchange integration support is now upgraded to support MS Exchange 2010. Java and Solr integrations are improved as well. Finally, we get hotfix notifications and one touch upgrade directly within the administrator. I'm just getting started.

ColdFusion has always been known for making the hard things easy, and it looks like ColdFusion 10 is no exception. Adobe's engineering team listened to customer feedback, giving us a lot of new toys to play with, including some great language enhancements. Highlights include important new array functions, app specific in-memory file system, MIME type checking on file uploads, an implicit CFC constructor, method chaining, and much, much more.

Oh, and closures. Did I mention closures? Not sure how this one will play out, yet, but I look forward to seeing what people come up with. There's also websockets, media player changes, and an entirely new charting to work with. Just scratching the surface here.

Yes, there is a lot of new stuff. And, to top it all off, Adobe placed ColdFusion Builder 2.01 in Beta at the same time. There are a lot of bug fixes in here, along with the CF 10 support and more. Did I mention both the server, and the IDE, have some HTML 5 goodness baked in as well? No? You'll just have to download them and start playing. This'll be fun.

MSOC Part 9: Application Event Handlers, The Rest

So far we've spent a lot of time going over the different Application Event Handlers that relate to specific scope events (application, session, and request). Some ask "How does this relate to MSOC?", to which I have to say "Because it's important to understand application flow and control." We'll be diving into the MSOC specific bits much more in the coming posts. For now, let's wrap this part of our MSOC talk with a brief discussion of our two final Application Event Handlers: onMissingTemplate and onError.

The onMissingTemplate handler kind of does what it says. ALERT: Pay Attention To This Next Statement. This handler will be automatically invoked when a ColdFusion template or component is called but does not exist. Did you catch that? Yes, this only catches requests for templates/files that might be served by the ColdFusion application server. Setup a special mapping for CF to process .inc files? Yes, if the file doesn't exist, then this handler should catch that. A .html page was requested? Unless the server is mapped for CF to handle those requests, then no, your onMissingTemplate method will not catch that error. You get the picture yet?

The basic layout of the onMissingTemplate method is like this:

view plain print about
2 * FUNCTION onMissingTemplate
3 * Runs when a (CF) template is called that does not exist
4 * @access public
5 * @returnType boolean
6 * @output false
7 */

8function onMissingTemplate(required string targetpage) {
9    // something goes here
10    return true;

As you can see, this method is automatically passed one argument, the page being requested. Here's another catch for you, this will only catch requests for ColdFusion processed files in the same directory as your Application.cfc. So a request for mysite.com/nothere.cfm will be caught, but a request for mysite.com/some/random/path/to/nothere.cfm will not (This surprised me too, and I had to test it over and over again to make sure I wasn't missing something). Weird, huh? Well, that's ok. To be honest, the onMissingTemplate handler is better served by those who don't have access to change their own webserver. The better option is to setup a 404 handler in their web server configuration. In the Apache Web Server you can look for the ErrorDocument line, in your httpd.conf file, and set it to a specific template that you setup for error handling.

view plain print about
1ErrorDocument 404 /errors/404.cfm

Speaking of error handling... You do do error handling, right? You wouldn't just show end users hard errors, would you? Of course not. You've got try/catch, throw and rethrow all over the place, right?

I've never seen an app that was made to handle every possible scenario for what a user could do. Nope, never. And, I've never seen (or written) a perfect app. It's a mythical creature, like the unicorn, or honest politicians.Fortunately, Coldfusion gives us multiple levels of error handling capability, from the block level of try/catch to the server level error handler applied through the CF Administrator. The onError method allows us to apply an application level error handler, that catches anything not previously coded for (through try/catch). It can display (if desired) any error thrown from onApplicationStart through onRequestEnd, including errors thrown through the process of the request itself, but can not display content for errors thrown in the onSessionEnd or onApplicationEnd.

The onError event handler is formed like this:

view plain print about
2 * FUNCTION onError
3 * This is an application wide error handler. Best practice would be to
4 * write process specific error handling, but this method will help
5 * you trap unexpected errors for custom notification and process
6 * @access public
7 * @returnType void
8 * @output true
9 */

10function onError(required exception, required string eventname) {
11    if(StructKeyExists(APPLICATION.cfc, "errorHandler")){
12        // If the error is thrown in onApplicationEnd or onSessionEnd, the error processor
13        // will still run, but nothing will be displayed to the user
14        WriteOutput(APPLICATION.cfc.errorHandler.process(argumentCollection: ARGUMENTS));
15    } else {
16        WriteDump(var = ARGUMENTS.exception, abort = true);
17    }

The onError method is automatically passed to argments, exception (think CFCATCH) and eventname (the application event that the error was thrown in). In my example, I've used an application wide error handler to process the error, and displayed it's output (if not thrown from onSessionEnd or onApplicationEnd). My handler could do any number of things, from logging to emailing the site admin to generating error code specific text response for output, to all of the above and more.

OK, that's it for the Application Event Handlers. Let's be honest, I really only glossed over them and they still took up a lot of posts. That said, it's time to get back to the core MSOC talk, even as important as understanding all of this might have been. We have things to talk about, like directory structures, shared and specific asset handling, extended application models, and more. Give me your feedback: What are your most pressing questions on running Many Sites with One Codebase?

The Joys of Developing for Internet Explorer

Note: Follow the madness here to it's conclusion, to discover yet another "Really? I didn't know that..." IE moment, that may save you heartache and pain.

Is the sarcasm evident in my title? It should be. While Internet Explorer may have been "groundbreaking" when it was released, it has ever been the bain of the web developer's existance. Why, you ask? Because it refuses to adhere to standards, and just operates differently than everyone else. To be fair, Internet Explorer 10 is in the works, and supposedly closes the gap a fair amount (and even wins out in some of the html 5/css 3 support), and 7, 8 and 9 did progressively improve (if slowly). Unfortunately the fact remains that some companies/organizations/governments are stuck on Internet Explorer 6, or maybe even 7. If you are a web developer, and write cross-browser web applications, chances are Internet Explorer has hit you more than once in your career.

It's the dumbest things that hit you too. The most obscure "WTF!?!" moments that drive you crazy. That is a daily experience for me now.


Intro to jqGrid Part 5: Search

At this point we've created a basic grid, filled it with data, refined the display of our columns and added event handlers to handle multiselect options. We've used custom cell formatters, used a custom datatype function, and even added and populated a toolbar in the process. Now let's start looking at some things that aren't necessarily jqGrid specific, but incorporate them for use in our grid. How about a search?

jqGrid includes some things for doing data search, that will automatically build modal windows and stuff. But sometimes you want to format things your own way, or incorporate jqGrid for use within an existing interface. One of the advantages for us, using the datatype function, is that we can preprocess our postdata prior to the ajax request.


Anatomy of a Shopping Cart: A Usability Study

This little writeup is a usability study of cart layout and process in general. So many apps today are still sporting the 1999 click-and-reload interface, and times have changed. Users are tired of the old way, embracing the new, and if you're behind the times you could be losing clients fast.

Shopping Carts are funny, terrible things. The less intuitive they are, the higher the abandon rate, and yet you have to pack a ton of stuff in there. The trick is to anticipate user workflow, and match that as much as possible, without making the user work for it too much. This is the whole precept behind Don't Make Me Think.

First things first, create a method for detecting JavaScript prior to a user ever coming to the shopping cart, as it redefines the experience for the user entirely. If JavaScript isn't present, then it's either a bot, some non-Class A mobile device, or less than 1% of the desktop user. Surfing the web without JavaScript is even less of a chance than a user surfing without the Flash player. There's just too much of a user experience that is lost without it, and most standard users don't have a clue how to disable it to begin with.

You end up with two separate cart processes, one with JavaScript, and one without. "Two processes?" Well, you write a process with graceful degradation, that can do what needs to be done in either scenario, and with code reusability in mind. This writeup will go over the user with JavaScript enabled.

I've mocked up a simple, yet effective, cart design. This is just a wireframe model, but there's nothing here that can't be done with HTML and CSS. A user without JavaScript would have to step through the stages of placing an order, screen by screen. A user with JavaScript has to step through as well, but through JavaScript and Ajax, this user does most of their work right from the cart.

Below is the mockup, with an explanation of the different stages of process to follow:

This is pretty loose. Balsamiq only allows you to do so much, so color and highlights and icons give things a lot more pizazz, but this should give you a good idea. If you're unfamiliar with Balsamiq, it's a great tool for doing simple "pen on napkin" wireframe layouts during interface design. It's also a great way to draft quick mockups for clients without spending hours of unpaid time, and even more hours on revisions.

From the top down:

The only button at the top is the "Keep Shopping" button. A user has to verify all the details prior to checking out, so the "Checkout" button is at the bottom of the form. The "Clear" button is also at the bottom, giving the user a second chance not to clear out. A second "Keep Shopping" button is at the bottom of the page, to encourage the user to buy more.

Next up is the cart contents. A user, coming to the cart, wants to know what's in it first. This is the user's main focus, so registering for the site or entering credit details and stuff always comes after the cart itself.

There are a few different ways to handle actions on cart items. Here I show the 'Remove' checkbox. If a user hits the box, take the item out. Don't prompt 'Are you sure?' This annoys users. Of course they're sure, or they wouldn't have checked the box. Another way of handling 'Remove' is part of an action link list. There are already two other action links showing (Gift Wrap and Add to Registry), it would be easy to adjust the layout to use all action links instead (Remove at the top, Add to Wishlist, Add to Registry, with Gift Wrap last). Here's a tip though: Any of these actions will ultimately remove the item from the cart.

Being tabular in nature, it's best to use a table here. Unwritten rules of cart usability: the 'Item' is listed first, unless something small, like a checkbox, radio button, or icon precedes it. The cost (Qty * Price) is always last. This allows us to build a columnar layout of the money details. If you have line item discounts, you may show the math here too, to emphasize what the client is getting. In this mockup, it's all shown below.

Next, show the user some love. Discounts, Promos, Gift Cards and Coupons should follow the cart. 1) it makes it easier to calculate the Subtotal and 2) it gives the user a warm fuzzy to watch their Total decrease. If there were discounts on items, pull them out here (if you didn't do it above). Don't make a user refresh the page to get a new Total. Gift Cards and Coupons? Use Ajax to verify their validity and value, and apply it to the tally block right then.

If addresses aren't pre-populated, then the user hasn't logged in or registered or set up addresses yet. Our action depends on which scenario is the case. If the user hasn't logged in or registered, then we show them a login box, with a register button. If they login, we go ahead with the rest of the order process. If they must register, handle that registration in a modal popup and then update your area. If the system allows for anonymous users (those who don't want, or have, to register with the site), or a registered user hasn't setup addresses yet, then we display "add address" links to the display that can load modal forms to collect the necessary details.

You can't tally Tax unless you know you need to collect it. If you don't have an address yet, don't show the Tax line item. If you get an address, and tax does need to be applied, add it in right away, and highlight it, and the Total change, so there's no surprises.

And, once you have an address, then you already know the zip/postal code the order is going to. Once a Shipping Method is selected we can lookup the shipping cost via Ajax. And, if the purchase is a download, then there's no reason to show them the Shipping line item at all. Avoid the "Estimated Shipping" thing, just give them the info once you have it. If need be, show them text ("We need your address and Shipping Method to show Shipping Cost") to help push them through the order process.

Last is the Payment Method. When a user selects the Payment Method, then the input fields for this area should change to the appropriate fields for that method. Don't show them fields for every possible option, just those for the option they've chosen.

That's it. By streamlining your Cart and Checkout process to match user workflow, you should see a lower abandon rate and higher sales. There's a lot of code involved, and both client-side and server-side data validation is a must, for security purposes, but the end gain makes the effort more than worthwhile. What are your suggestions for improving Cart usability? Give me your feedback below.

Previous Entries / More Entries