Cutter's Crossing

ColdFusion 10 is Now Available

Tue, 15 May 2012 07:58:00 -0400

Adobe has released ColdFusion 10. The engineering team has done a fantastic job putting this release together. Here's a short list of some of what's new:

Improved/Updated
- caching
- ORM
- webservices
- scheduler
- hotfix management
- security
- MS Exchange integration
- security
- Solr
- charting (now html 5)
- Java integration
- Flex and AIR lazy loading
- VFS
- more!
html 5 support
websockets
RESTful services
greater cfscript parity
geo-tagging
CFC auto constructors
method chaining
closures
Tomcat (no more JRun)
more!

ColdFusion Builder

CFQueryReader v2.1: Now with metaData support

Wed, 09 May 2012 10:16:00 -0400

The Sencha guys just keep upping the bar, and Ext JS 4.1 is no exception. I've been reading Loiane Groner's Ext JS 4 First Look, to review it, and continually find new, cool stuff. 4.1, however, takes it even further. While upgrading CFQueryReader, I was working with 4.1 RC2. And, while extending the base classes, I came across a new feature that wasn't fully documented yet: adding metaData to a server-side store response for changing configuration on the fly. Sometimes it would be nice to just...change up. Now that 4.1 is fully released, I had to make sure that this worked in CFQueryReader. So, last night I sat down and hammered out this functionality. It took a lot more than I realised, and I learned a lot more about the Ext JS internal code, but I think CFQueryReader is better for it. Consider the following ColdFusion method:


/**
 *	FUNCTION getWithMeta
 *	This function returns the ColdFusion Query object as part of a struct object.
 *
 *	@access remote
 *	@returnType struct
 *	@output false
 */
function getWithMeta(numeric pageIndex = 1, numeric pageSize = 50, string sort = "", string search = "") {
	var retVal = {"success" = true, "pageIndex" = ARGUMENTS.pageIndex, "pageCount" = 0, "recordCount" = 0, "message" = "", "getEntries" = "", "metaData" = {"root" = "getEntries", "totalProperty" = "recordCount", "successProperty" = "success", "messageProperty" = "message", "idProperty" = "id", "fields" = []}};
	StructAppend(LOCAL.retVal, GetEntries(argumentCollection: ARGUMENTS), true);
	var colArr = ListToArray(LOCAL.retVal.getEntries.columnList);
	LOCAL.retVal.metaData.fields = [
		{"name" = "id", "type" = "string", "mapping" = JavaCast("int",0)},
		{"name" = "title", "type" = "string", "mapping" = JavaCast("int",3)},
		{"name" = "posted", "type" = "date", "mapping" = JavaCast("int",2)},
		{"name" = "views", "type" = "int", "mapping" = JavaCast("int",1)}
	];
	return LOCAL.retVal;
}

I reused my getEntries method, to get my query. Here, I'm creating a metaData object, at the root of the return, to define the dataset. Configuration options that I normally define client side (root, totalProperty, etc) I put in to the metaData key. When the response is received by the client, the reader will pass this metaData in to our app, applying this configuration to our reader, store, model, and so on. In the above method, we let the metaData map our columns to fields, rather than doing it client side (CFQueryReader will automatically skip the column mapping if metaData.fields is present in the response.) Our client side store might now look like this:


Ext.create('Ext.data.Store', {
	storeId: 'entryStore',
	model: 'Entry',
	remoteSort: true,
	proxy: {
		type: 'ajax',
		url: '/com/cc/Blog/Entries.cfc',
		extraParams: {
			returnFormat: 'json',
			method: 'getWithMeta'
		},
		limitParam: 'pageSize',
		pageParam: 'pageIndex',
		sortParam: 'sort',
		reader: {
			type: 'cfquery'
		}
	},
	autoLoad: true
});

A full example of this, in action, can be seen on a demo page of the CFQueryReader site. The full source code, of the example, can be found in the CFQueryReader GitHub repository.

Ext JS 4.1 Grid: Part1: Basic Config

Mon, 07 May 2012 23:00:00 -0400

Many moons ago, I wrote a series on My First Ext JS Data Grid. It was very popular. In fact, it got me the gig co-authoring two books on Ext JS and spawned an open source project targeted at integrating Ext JS with ColdFusion. But, I did that series back in 2007, using Ext JS 1.1 (maybe?), and an update is long overdue.Today I'll start that update. Well, actually it began almost two weeks back, when I announced the update to CFQueryReader. In that post, and in the demo code in the GitHub download, I show you how to define a data Model and a data Store, using CFQueryReader as a custom data Reader. What isn't in the post (though it is in the GitHub download), is how easy it is to create a data grid with Ext JS 4.1


Ext.create('Ext.grid.Panel', {
	title: 'Testing: getWithMeta (query as part of custom struct)',
	store: Ext.getStore('entryStore'),
	renderTo: 'demo',
	columns: [
	    {header: 'ID', dataIndex: 'id'},
	    {header: 'Title', dataIndex: 'title'},
	    {header: 'Posted', dataIndex: 'posted'},
	    {header: 'Views', dataIndex: 'views'}
	],
    width: 380,
    height: 600
});

The really important pieces here are the store, renderTo, and columns configuration options. The store references the storeId of the data store you create to feed the grid. The renderTo references the id of a div element to render the grid within, on your page. The columns option defines, in order, the columns to display within the grid. In this configuration we have the minimum information needed: the header title of the column and the data field the column will map to. That's it! Oh, sure, there's a lot more you can do: paging bars and column renderers and grouping and stuff. We'll get into it. But, for now, this is all you need to create a very basic grid in Ext JS 4.1. You can download the demo code from the CFQueryReader GitHub repository.

ColdFusion Query Column Order: Did You Know?

Sat, 05 May 2012 18:48:00 -0400

Now first let me say that I've only seen this behavior on ColdFusion 9.01 with Cumulative Hotfix 2. My ColdFusion 10 VM blew up this morning, and I haven't had 7 or 8 for a while now, so someone else might have to verify this behavior on those platforms for me. This morning I was testing out a new feature of Ext JS 4.1, and ran into something interesting. One of the reasons that I wrote CFQueryReader, as well as serializeCFJSON, was because of the way that ColdFusion handles it's JSON conversions of it's native query object.


{
	"COLUMNS":["ID","NAME","EMAIL"],
	"DATA":[
		[1,"Ed Spencer","ed@sencha.com"],
		[2,"Abe Elias","abe@sencha.com"],
		[3,"Cutter","no@address.giv"]
	]
}

I've written before, about how Adobe's dataset serialization is much trimmer, but many client-side libraries and plugins expect a more standardized format.


{
    "users": [
       {
           "id": 1,
           "name": "Ed Spencer",
           "email": "ed@sencha.com"
       },
       {
           "id": 2,
           "name": "Abe Elias",
           "email": "abe@sencha.com"
       }
    ]
}

It's not too difficult for us, client-side, to map the column to the proper data position in the record array. In fact, that's basically what CFQueryReader and serializeCFJSON both do. But why is it even necessary? Most of these client-side libraries allow us to write mappings in configurations anyway, so why write something else to do it? Well, there's two reasons really. First of all, the client-side developer may not have that much insight into the server-side developer's code (they can be different people), and would probably be unaware of API changes. The second reason? Well, if you've used ColdFusion for a while, you might know that there was a time when you could not depend on the column order always being the same. At one point, the columns were returned in alphabetical order. This would make it really hard to figure the position out prior to run time, unless you have intimate knowledge of every change to your API as it happens. It appears that this isn't as much of a problem today. Kind of. I'll run this query:


LOCAL.sql = "SELECT	SQL_CALC_FOUND_ROWS id,
		views,
		posted,
		title
FROM	tblblogentries
WHERE 0 = 0";
LOCAL.q = new Query(sql = LOCAL.sql, datasource = VARIABLES.dsn);
LOCAL.retVal.getEntries = LOCAL.q.execute().getResult();

If you dump the results to the page, you'll see the following:

Now, server-side, ever query comes with a variable we can use to reference the columns used in a query. I'll use ColdFusion's ArrayToList(LOCAL.q.columnList) method to dump the query's columnList property to the page:

Look closely and you'll see that both appear to be in alphabetical order. The query dump and the column list showed them in alphabetical order. That said, we then look at the JSON return:

Wait a minute? Now the columns appear in the same order that they are in the query. That's a change.Let's change the column order in the query, and see if it changes anything.


LOCAL.sql = "SELECT	SQL_CALC_FOUND_ROWS id,
		title,
		posted,
		views
FROM	tblblogentries
WHERE 0 = 0";

Notice that, again, the column order matches that of the order it was used in the query. Running a cfdump of the query, and it's columnList, still show alphabetical order. Why is it different? Why does this matter? Well, I'm going to show you that in a follow-up post (on Ext JS 4.1 and ColdFusion), but in the mean time it's a good thing to know, that the JSON returnFormat will give you query column orders identical to that used in the query. And, as I said above, somebody please verify this for me on other versions of the platform.

JQuery Plugin: serializeCFJSON

Wed, 02 May 2012 22:41:00 -0400

Quick note about my serializeCFJSON project out on GitHub. I wrote this quick JQuery plugin to convert ColdFusion's JSON representation of it's native query objects. ColdFusion represents datasets in a trim manner, as an object containing two arrays: One, an array of column names, and the other, an array of arrays, each representing one record of the set. Most representations of recordsets are an array of objects, each record represented as a collection of name/value pairs. ColdFusion's representation is much smaller, removing a lot of unnecessary duplication, but many pre-built frameworks and plugins look for the name/value pair objects. What the plugin does is convert ColdFusion's representation into the more standard form. Ajax transfer is unaffected, as the trimmer format is still being passed. What we get is some very minor client-side overhead in the creation of a new object. What's more, the plugin recursively searches through a JSON object and converts any ColdFusion query that it finds. So, if you nest your query inside a larger object, the plugin will still convert it for you. I put up a demo in the Projects menu, at the top of this site. This is a refactor of my Grouped jqGrid example, using the plugin for the data conversion. The full sample code for the demo can be found in the GitHub repository. Take her out for a spin, and let me know what you think.

CFQueryReader 2.0: Site and Demo Updates

Thu, 26 Apr 2012 01:09:00 -0400

Note: I mistakenly posted this under the wrong title. I must get more sleep ;) I finally got around to a major overhaul of the CFQueryReader site, including all new demos and documentation of the latest build for Ext JS 4.x. The new demos for Ext JS 2.x and 3.x include links to legacy API's for both versions of the library. The 3.x version includes an example of using Ext Direct, and the new 4.x demo includes paging data grids. (Big thanks to Loiane's Ext JS 4 First Look, which showed me very quickly how dead simple Ext JS grids and data stores have become.) I have not yet tested CFQueryReader with Sencha Touch, but it should work. Any feedback just let me know.

Ext JS 4 and ColdFusion: CFQueryReader 2.0

Mon, 23 Apr 2012 16:22:00 -0400

Wow! It's been too long. I've been so buried playing with new stuff I haven't had much time to write about it. Time to rectify that. My position over the last year and a half hasn't required me to use Ext JS. And without a project I really haven't had much time to dive in to Ext JS 4. But, I am reviewing Loiane's Ext JS 4 First Look right now, and decided the simplest way to get in was to apply what she was telling me. I knew some of the basics, but Ext JS 4 is a big change, and Loaine's book quickly helped me to grok differences in the class model, updates to the data api, and more. In no time at all I had completely rewritten CFQueryReader, for parsing the native JSON return of a ColdFusion query object. I need to verify, but I'm fairly certain there are more lines of comments than there are code, in the new file. And it is wicked simple. First, include the js file, then define a record of data:


	Ext.define('Artist', {
		extend: 'Ext.data.Model',
		fields: [
			{name:'city', type:'string'},
			{name:'state', type:'string'},
			{name:'postalcode', type:'string'},
			{name:'email', type:'string'},
			{name:'phone', type:'string'},
			{name:'fax',type:'string'},
			{name:'thepassword',type:'string'},
			{name:'artistId', type:'int'},
			{name:'firstName', type:'string'},
			{name:'lastName', type:'string'},
			{name:'address', type:'string'}
		],
		idProperty: 'artistId'
	});

Then, define your data 'Store':


	Ext.create('Ext.data.Store', {
		model: 'Artist',
		storeId: 'artistStore_1',
		remoteSort: true,
		proxy: {
			type: 'ajax',
			url: '/com/cc/ArtGallery/Artists.cfc',
			extraParams: {
				method: 'GetAllStandard',
				returnFormat: 'json'
			},
			reader: {
				type: 'cfquery'
			}
		},
		autoLoad: true
	});

That example is for a method that only returns a query. I like to return a struct, so I also included the ability to reference that:


	Ext.create('Ext.data.Store', {
		model: 'Artist',
		storeId: 'artistStore_3',
		remoteSort: true,
		proxy: {
			type: 'ajax',
			url: '/com/cc/ArtGallery/Artists.cfc',
			extraParams: {
				method: 'getAllInStruct',
				returnFormat: 'json'
			},
			reader: {
				type: 'cfquery',
				query: 'getArtists',
				totalProperty: 'totalRowCount',
				successProperty: 'success',
				messageProperty: 'message'
			}
		},
		autoLoad: true
	});

It's really that simple! Go pull CFQueryReader from RIAForge and test it out against the demo CFArtGallery datasource included in your CF install.

Intro to jqGrid Part 7: Grouping

Wed, 04 Apr 2012 17:02:00 -0400

OK, I know I said I was done, but I kept playing around and figured I should share. jqGrid, like other (read: Ext JS) good grid implementations, has the ability to group your data. What does that mean exactly? Well, let's look at our demo for inspiration. We've created a grid of blog post entries. Entries have associated Categories. We can reconfigure our grid to show the entries grouped by Category, with Category headers and accordion like separation.Accomplishing this is fairly easy, taking a combination of jqGrid configuration, and changes to our data retrieval process. First we'll look at our configuration. We need two new config attributes: grouping and groupingView.

jqGridDemo.js - Grid Configuration


grid.jqGrid({
	...
	sortname: 'Posted',
	sortorder: 'asc',
	postData:{method:"GetGroupedEntries",returnFormat:"JSON"},
	grouping: true,
	groupingView: {
		groupField: ['categoryName'],
		groupDataSorted: true
	},
	...
});

Simple enough. We said "Yes, we want a grouping grid", and then told it which column to group on. "Why is the groupField an array?" Good question. The writers of jqGrid are planning to support multiple grouping levels in the future, and will use this array form to define multiple columns at that time. Since jqGrid can only take one column right now, just place that top level grouping column. We also set groupDataSorted to true, so that it will pass the groupField column name as part of the sortCol argument on server-side requests. This means that all server-side requests will now have a sortCol in the form of {groupField} asc, {sortCol}. The need for this will become apparent when we adjust our paging query. Let's look at that next. You'll notice I adjusted the postData attribute, and changed the name of the method we'll call. I also changed our initial sorting column and direction. In our Entries.cfc, let's copy our GetEntries method, paste it again, and rename the new one to GetGroupedEntries. From there, let's start off by looking at our query. We've never pulled the Categories before, so now we need to adjust to pull in the necessary information. Because there can be multiple categories assigned to an entry, BlogCFC uses a mapping table, tblblogentriescategories, to link entries (tblblogentries) to categories (tblblogcategories). By adding some JOIN statements, and table aliases, we can now add our column to the query.

Entries.cfc - GetGroupedEntries Query


LOCAL.sql = "SELECT	SQL_CALC_FOUND_ROWS b.id,
				b.title,
				c.categoryname,
				b.posted,
				b.views
			FROM	tblblogentries b
			INNER JOIN tblblogentriescategories bec ON bec.entryidfk = b.id
			INNER JOIN tblblogcategories c ON c.categoryid = bec.categoryidfk
			WHERE 0 = 0
			 ";

Then we're required to have our records grouped by category. This is why jqGrid adjusted the sortCol variable, because we need to add the groupField to our ORDER BY clause. You'll recall, though, that we had a bit of code to verify the validity of our sortCol variable. We'll need to adjust this first, or it will throw an error.

Entries.cfc - GetGroupedEntries: Set ORDER BY


LOCAL.scArr = ListToArray(ARGUMENTS.sortCol);
LOCAL.sortCol = (ArrayLen(LOCAL.scArr) eq 2) ? LOCAL.scArr[2] : ARGUMENTS.sortCol;
// Verify that your sort column and direction are valid. If not, then return an error.
if(ArrayFindNoCase(VARIABLES._COLUMNARRAY, Trim(LOCAL.sortCol)) AND ArrayFindNoCase(VARIABLES._DIRARRAY, ARGUMENTS.sortDir)){
	LOCAL.orderby = ARGUMENTS.sortCol & " " & ARGUMENTS.sortDir;
} else {
	StructAppend(LOCAL.retVal,{"success" = false, "message" = "Your sort criteria is not valid."},true);
	return LOCAL.retVal;
}

What are we doin' here? Well, we convert sortCol argument to an array, and create a LOCAL.sortCol variable. If our array has two items, we set our local variable to the second item. Otherwise we just use the argument. We then test the local variable for validity (is this a valid sort column?), and set our LOCAL.orderby. Now we only need two minor adjustments to our javascript, to accomodate the new column. The first is in the colModel in the grid config.

jqGridDemo.js - Grid Configuration: Column Model


grid.jqGrid({
	...
	colModel: [
		{name: 'Action', index: 'ID', label: 'Action', width: 60, fixed: true, sortable: false, resizable: false, align: 'center', formatter: 'actionFormatter', key: true},
		{name: 'Title'},
		{name: 'Posted', label: 'Release Date'},
		{name: 'Views', align: 'right', width: 60, fixed: true},
		{name: 'categoryName'}
	],
	...
});

The last bit of script is to add the column in our bits that map the ColdFusion return to the colModel.

jqGridDemo.js - populateGrid: Column Mapping


grid.jqGrid('setGridParam',{remapColumns:[
	gridCols['ID'] + gridMultiSelect,
	gridCols['TITLE'] + gridMultiSelect,
	gridCols['POSTED'] + gridMultiSelect,
	gridCols['VIEWS'] + gridMultiSelect,
	gridCols['CATEGORYNAME'] + gridMultiSelect
]});
grid[0].addJSONData(d);

Now we can load our template in the browser to see the result.

Great! We now have grouped data. We can page through, and see where new categories start. You'll notice that our total record count is much higher than before. That's because there's a one-to-many relationship between entries to categories, so entries are repeated in each category that they are a part of. This is great, but the first thing I notice is that the new column actually displays in the grid. Since the category is already the grouping header, I don't think we need to show it in the grid columns. Luckily there's an option to the groupingView that will allow us to hide the column, groupColumnShow.

jqGridDemo.js - Grid Config: groupingView


grid.jqGrid({
	...
	grouping: true,
	groupingView: {
		groupField: ['categoryName'],
		groupDataSorted: true,
		groupColumnShow: false
	},
	...
});

This hid the column but, as you can see, it also changed the size of the grid when it removed the column. This is a bug in jqGrid, and is filed with the development team, but in the meantime you can add some code to handle this issue. Our gridComplete configuration attribute is already setup to call our gridLoadInit method, which is called every time the data loads. We'll add a line at the end of this method to repair our grid width.

jqGridDemo.js - gridLoadInit Method


var gridLoadInit = function () {
	...
	grid.jqGrid('setGridWidth',800);
};

If we reload our page again, we'll see that the grid width is now as it should be, with the column being hidden as intended. What's next? Each grouping section header has an icon, showing the user that the sections can be collapsed or expanded. In our current configuration (the default) all sections are expanded by default. Let's change that so that they're all collapsed on load. This is easily accomplished by adding groupCollapse to our groupingView configuration. We'll also change the classes of the expand and collapse indicators used, so that they match up with the rest of our demo. We're using the FamFamFam Silk icon library, so we'll reference some images from that.

jqGridDemo.js - groupingView Configuration


groupingView: {
	groupField: ['categoryName'],
	groupDataSorted: true,
	groupColumnShow: false,
	groupCollapse: true,
	plusicon: 'bullet_toggle_plus',
	minusicon: 'bullet_toggle_minus'
},

The plusicon and minusicon attributes denote class references. We'll create these in our stylesheet.

jqGridDemo.css - icon references


/* bullet_toggle_minus icon image for trigger */
.bullet_toggle_minus { background: url('/resources/images/icons/bullet_toggle_minus.png') no-repeat scroll 0px 0px transparent !important; }

/* bullet_toggle_plus icon image for trigger */
.bullet_toggle_plus { background: url('/resources/images/icons/bullet_toggle_plus.png') no-repeat scroll 0px 0px transparent !important; }

.ui-jqgrid tr.jqgroup td { font-size: 1.1em; font-weight: bold !important; background-color: #98AFC7; }

When you reload your page you'll see that the groupings are now collapsed by default, and notice the new icons. I even added some style stuff to differentiate the headers from the data a little more clearly. Now, personally, I would prefer that all of the sections are collapsed on load, except the first one. We can do this by adding a line to our gridLoadInit method.

jqGridDemo.js - gridLoadInit Method


var gridLoadInit = function () {
	...
	grid.jqGrid('groupingToggle','gridTestghead_0');
	grid.jqGrid('setGridWidth',800);
};

The groupingToggle method is used to open or close a grouping session, taking the row id as the argument. Grouping headers are just another table row, each with it's own unique id. It's format is {grid id}ghead_{section index}. Our grid id is gridTest, and JavaScript uses 0 based indexes, so the first section is index 0. Placing this in the gridLoadInit, a request for data is made, the data is applied to the grid in groups, the groups are collapsed (and the column is hidden) by configuration, the load method is called, then the first section is expanded, and the width is set on the grid. Whew! That's a lot of stuff. It now looks like this.

This is coming along very nicely. There's just one final thing I'd like to explore here, summary rows. One common function of grouping is to have summaries of specific columns: calculations (or functions) run on each record of data in a group to arrive at some 'summary' value. For instance, you might have some sort of user logging setup, and want to show the average number of page views per user. Or an e-commerce control panel, where you are grouping user orders, and want to show the total amount that's been spent by clients across all of their grouped orders. In our demo we show how many times each post has been viewed, so we'll break it down to the total number of views per category. We begin by adjusting our column model, to say we want to tally up the views.

jqGridDemo.js - jqGrid Config: Column Model


grid.jqGrid({
	...
	colModel: [
		...
		{name: 'Views', align: 'right', width: 60, fixed: true, summaryType: 'sum'},
		...
	],
	...
});

There are several different summaryTypes that may be applied to a column.

sum - apply the sum function to the current group value and return the result
count - apply the count function to the current group value and return the result
avg - apply the average function to the current group value and return the result
min - apply the min function to the current group value and return the result
max - apply the max function to the current group value and return the result

You can also apply a custom function, or even a template, to a column. For more information see the jqGrid wiki entry on Grouping. I don't have enough space in the Views column to put a label on that total, so I need something in the Release Date footer. Since I'm not really creating a true summary of that column I need to apply an empty method to the summaryType, then apply my output to the summaryTpl like this:

jqGridDemo.js - jqGrid Config: Column Model


var emptyMethod = function (){
	return;
};

grid.jqGrid({
	...
	colModel: [
		...
		{name: 'Posted', label: 'Release Date', summaryTpl: '

Total Views: <\/div>', summaryType: emptyMethod},
		...
	],
	...
});

After that I adjust my groupingView to give me the summary:

jqGridDemo.js - jqGrid Config: groupingView


groupingView: {
	groupField: ['categoryName'],
	groupDataSorted: true,
	groupColumnShow: false,
	groupCollapse: true,
	plusicon: 'bullet_toggle_plus',
	minusicon: 'bullet_toggle_minus',
	groupSummary: [true]
},

Then I'll add a little styling, to make it stand out, and align my summaryTpl output.

jqGridDemo.css


div.dateSummaryFooter { text-align: right; }

tr.jqfoot td { background-color: #736F6E; color: #FFF; }

When you refresh the page you'll see how it all lays out.

As you can see, grouping in jqGrid is pretty easy to do. There are more options to refine your grouping view as well. See the documentation for more. As always, I welcome your feedback. Sample code can be obtained from the Download link below.

ColdFusion Security Hotfix and Big Forms

Tue, 27 Mar 2012 14:41:00 -0400

The other day, Adobe released a new Security Hotfix for it's ColdFusion server. There were a number of things addressed in the hotfix, to help protect against Denial of Service attack using a hash algorithm collision. (My wife would say I sound like Charlie Brown's teacher right about now.) Ok, the important thing is you need to update your server. Now for the fun part. We loaded the fix to our testing servers to run our app around the block prior to pushing this up to production. And, it's a good thing we did. We're preparing for a large deployment, and testing is pretty heavy right now. First thing in is that a form would no longer submit, throwing a 500 error every time. I didn't show me a 500 error, just a blank page. I had to look at Firebug to see the error code response. Now, if you've ever encountered a 500 error from the server then you know they don't typically tell you much. I reproduced the error locally and then went looking through the log files on the server. In a multi-server configuration there are two core areas to look at log files. The first are the basic JRun logs. On a Windows systems, these files are located in the C:\JRun4\logs folder. Here you will typically find a {instance}-out.log file, and a {instance}-event.log file, for each ColdFusion instance you have configured. Right out of the gate you have admin-event and admin-out logs for the JRun administrator, and cfusion-event and cfusion-out for the default ColdFusion instance. I checked both files for my instance, and saw there weren't any items to tell me about the 500 error, so I then went looking at the ColdFusion logs. Each ColdFusion instance has it's own set of log files, that you can see in the logs viewer in the ColdFusion Administrator. That said, the ColdFusion Administrator is not really the best place to go through these files, especially when you're really having issues. At this point, you just want to open them yourself. First, you have to find them. You do this through your instance, C:\JRun4\servers\{instance}\cfusion.ear\cfusion.war\WEB-INF\cfusion\logs. You'll probably find multiple log files here, from the application and eventgateway and mail logs, to individual logs from cflog calls. Our issue, with submitting our form, was answered by the JRun -event logs, which gave me a few error messages saying something like this:


03/27 07:52:00 error ROOT CAUSE: 
coldfusion.filter.FormScope$PostParametersLimitExceededException: POST parameters exceeds the maximum limit specified in the server.
	at coldfusion.filter.FormScope.parseQueryString(FormScope.java:397)
	at coldfusion.filter.FormScope.parsePostData(FormScope.java:346)
	at coldfusion.filter.FormScope.fillForm(FormScope.java:296)
	at coldfusion.filter.FusionContext.SymTab_initForRequest(FusionContext.java:377)
	at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:33)
	at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
	at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)
	at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:126)
	at coldfusion.CfmServlet.service(CfmServlet.java:200)
	at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
	at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
	at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
	at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
	at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
	at jrun.servlet.FilterChain.service(FilterChain.java:101)
	at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
	at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
	at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
	at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
	at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
	at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
	at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

"Dude! What is that!?!" Well, luckily I had installed my security hotfix, locally, just that morning, so I remember reading over the instructions. In it's notes it had stated the following:

Customers who want to change postParameterLimit, go to {ColdFusion-Home}/lib for Server installation or {ColdFusion-Home}/WEB-INF/cfusion/lib for Multiserver or J2EE installation. Open file neo-runtime.xml, after line "100.0" add the below line and you can change 100 with desired number. "100.0"

Just a heads up, that neo-runtime.xml file is minified, so you'll want to Find "postSizeLimit" to get that statement in the right place. We tried that postParametersLimit value (100) and found that our form had more than that (many were hidden, but that's another post all together), so we adjusted the number to 200. After restarting the instance again, we tested the form once more with complete success. Hopefully this will help someone else avoid this issue. It's important to remember that Adobe does try to document these types of situations with hotfixes, so when you run into issues they should be your first source of troubleshooting information.

Intro to jqGrid Part 6: The Action Column

Fri, 16 Mar 2012 10:19:00 -0400

So far we've covered the base grid configuration, populating the grid with remote data, controlling layout by refining the column model, and given an example of event handling by binding a search form to our grid. This covers most of what the average developer might need to know about working with jqGrid, but I've been promising to tell you how to work with the icons we setup in our action column.jqGrid provides the ability to add predefined 'action' icons directly. That said, it's a small, predefined set of actions, and they build out dialogs and forms according to configuration details. This might be nice for prototyping, or small applications, but most of us already have interfaces for editing and stuff. I like finer control over the 'look' of my interfaces, and prefer to use stuff that's already built, if we have it. So, we make our own 'action' icons, and attach our own custom events. First, let's look again at the custom cell renderer that we wrote for our 'action' column:


$.extend($.fn.fmatter, {
	actionFormatter: function(cellvalue, options, rowObject) {
		var retVal = "";
		retVal += "";
		return retVal;
	}
});

Yeah, this one was pretty simple. All it did was add two icons to the display and apply the cell value to the rel attribute. The cell value, in our case, is the ID column, which is the unique reference you would want for this type of scenario. Now, the first thing you might notice, in reviewing the current output, is that something looks a little off. Can you spot it?

If you look close you can see it. Every other row the icons and text are a little washed out. This is because of the altRows option we used in our jqGrid configuration. Removing that option will remove the alternate row highlighting, which is causing the washout effect. Here, let's run a little experiment. First, remove the altRows attribute from our grid configuration. A quick reload will show you the washout is gone, just like I said. Now, let's create a new style declaration in our stylesheet.


table#gridTest tr:even td { background-color: #F5F5F5; }

If you reload the page you'll notice that didn't do anything. Why is that? Well, good question. Maybe it's because the rows don't exist until after the request is made. Let's take a different tact. Let's change that style declaration, making it define a class.


.evenRow { background-color: #F5F5F5; }

Next, let's add a line to the end of our gridLoadInit method.


$('table#gridTest tr:even td').addClass('evenRow');

Now refresh your grid. You get the grid highlighting without the washout effect. Cool! You might remember that, in a previous post, the gridLoadInit method is attached to the grid's gridComplete configuration option, and fires each time data is loaded into the grid. That method becomes very important, as it allows you to do things with the records/rows once they are loaded. "But, what does this have to do with the 'Action' column stuff?" Well, that leads to our first method of attaching event handlers to our action icons.

Bound Handlers

For a number of reasons, I'm using an older version of JQuery in these posts. You do have access to .live() binding, but sometimes it's nice to really take control of when you are binding/unbinding events. Basically, we want to apply event handlers to every row after it loads, and remove those bindings before we load new records. We'll start by creating a new method that we can call from within our gridLoadInit method. You could put all of the code directly in the method, but let's break it up a little to make maintenance a little easier. We'll write a new bindActionHandlers method inside our $(document).ready().


/*
 * FUNCTION bindActionHandlers
 * This method is called within the gridLoadInit() method, and is used to
 * bind event handlers to the action icons of new records
 */
var bindActionHandlers = function () {
	
};

Here we'll place all of our code to create new event bindings for our action columns. Let's start by binding a click handler to our delete icons.


var bindActionHandlers = function () {
	// Delete icon binding
	$('span.delete[class*="action-trigger"]', grid).bind("click", function(e){
		e.preventDefault();
		
		return false;
	});
};

What we're saying here is a) apply this method as the click handler to delete icons inside our grid that include the action-trigger class, and b) prevent any default click action and return false when we're done. Now, let's think about this a minute. What would we expect a user to see and do when they are trying to delete this record? Well, we probably want to make sure that they really want to remove the record. Let's build them a little confirmation dialog. First, we probably need to know which record they're trying to remove. We probably also want to show them the title of the record in the confirmation. Let's set up some function local variables.


var bindActionHandlers = function () {
	// Delete icon binding
	$('span.delete[class*="action-trigger"]').bind("click", function(e){
		e.preventDefault();
		var row = $(this).parent().parent(),
			rowId = row.attr("id"), 
			recId = $(this).attr('rel'),
			title =	grid.jqGrid('getCell',rowId,'Title');
            
		return false;
	});
};

OK, this gets us the id of the record, along with the title. Now we'll build a little confirmation box.


var bindActionHandlers = function () {
	// Delete icon binding
	$('span.delete[class*="action-trigger"]').bind("click", function(e){
		e.preventDefault();
		var row = $(this).parent().parent(),
			rowId = row.attr("id"), 
			recId = $(this).attr('rel'),
			title =	grid.jqGrid('getCell',rowId,'Title');
		// Create a dynamic dialog, that is destroyed when closed
		$('

').dialog({
			title:'Delete Confirmation',
			width:425,
			height:200,
			modal:true,
			create: function(){
				$('span.ui-icon-closethick').html("");
			},
			close:function(){
				$(this).dialog('destroy');
			},
			buttons:[
			    {text:'OK',
			    click:function(){
			    	deleteItem(rowId,recId,$(this));
			    }},
			    {text:'Cancel',
			    click:function(){
			    	$(this).dialog('close');
			    }}
			]
		}).html("Are you sure you want to delete the following?:
"+title);
		return false;
	});
};

Now we create a JQueryUI dialog on the fly, to use as a confirmation box. We setup OK and Cancel click handlers, and dynamically set the text of the dialog to reflect the title of the record being deleted. We also call adeleteItem() method, to delete the record. We'll build that later. Now that we've put all of this together, all we need to do is add a call to the bindActionHandlers() method to the end of our gridLoadInit() method.


var gridLoadInit = function () {
	// if the 'selected' array has length
	// then loop current records, and 'check'
	// those that should be selected
	if(selArr.length > 0){
		var tmp = grid.jqGrid('getDataIDs');
		$.each(selArr, function(ind, val){
			var pos = $.inArray(val, tmp);
			if(pos > -1){
				grid.jqGrid('setSelection',val);
			}
		});
	}
	
	$('table#gridTest tr:even td').addClass('evenRow');
	bindActionHandlers();
};

Now let's reload the page, and click the delete icon of our first row.

The downside to dynamically applying your event handlers like this is that you also need to unbind handlers before you refresh your data. We already stubbed out a method for this in our $(document).ready(), gridUnloader.


/*
 * FUNCTION gridUnloader
 * Called from within the populateGrid method, this method is to unbind
 * any event handlers created during grid load, by the gridLoadInit method.
 */
var gridUnloader = function () {
	
};

After that you just add the code to unbind the click handler


var gridUnloader = function () {
	$('span.delete:not("disabled-trigger")', grid).unbind('click');
};

This says unbind any click handler applied to a grid's delete icons that don't have a class of disabled-trigger. ("Hey Cutter! What's That!" All in good time...) Now that you have the method, all you have to do is call it from the correct point of context. Within the data grid config option, which handles all requests for grid data, we had a piece of code that runs on the first request (to map column positions). This is in a conditional that basically says "if not set, then set it". We put an else clause here, to now see "if it is set, then we're reloading the grid. We know if we're reloading we need to unbind all our row level event handlers, so this is where our call to gridUnloader will go.


var populateGrid = function (postdata) {
	$.ajax({
		url: '/com/cc/Blog/Entries.cfc',
		data: $.extend(true, {}, postdata, {search: $.toJSON(scrubSearch())}),
		method:'POST',
		dataType:"json",
		success: function(d,r,o){
			if(d.success){
				
				// If loading for the first time, let's find out to which
				// array positions our columns map.
				if(!gridCols.set){
					columnSetup(d.data);
				} else {
					// Unbind any bound event handlers in the current grid data
					// Only necessary after the first grid load
					gridUnloader();
				}

				grid.jqGrid('setGridParam',{remapColumns:[
					gridCols['ID'] + gridMultiSelect,
					gridCols['TITLE'] + gridMultiSelect,
					gridCols['POSTED'] + gridMultiSelect,
					gridCols['VIEWS'] + gridMultiSelect
				]});
				grid[0].addJSONData(d);
			} else {
				console.log(d.message);
			}
		}
	});
};

Let's stop and take a minute to build that deleteItem method, and the remote method it will call. There's a few things that need to happen.

Make an ajax call to remove the record
If the call is successful, remove the row from the grid
If the call fails, show a message to the user
Close the Delete Confirmation dialog box

Let's start by writing a quick dialog configuration object for our generic error dialog. We'll place this inside our $(document).ready() statement.


// Error dialog box config
$('div#grid-dialog-error').dialog({
	width:400,
	autoOpen: false,
	modal: true,
	create: function(){
		$('span.ui-icon-closethick').html("");
	},
	buttons:{
		"OK": function(){
			$(this).dialog("close");
		}
	}
});

We'll use this in a few places going forward. Right now, let's write our ajax call for removing a blog entry.


/*
 * FUNCTION deleteItem
 * This method is for deleting records and removing the
 * corresponding grid entry
 * @rowId (int) - ID of the row to be removed from the grid
 * @recId (string) - The UUID of the blog entry to be removed
 * @dlg (object) - JQuery element of the JQUI dialog
 */
var deleteItem = function (rowId, recId, dlg) {
	$.ajax({
		url: '/com/cc/Blog/Entries.cfc',
		data: {
			method: 'deleteEntry',
			recId:recId,
			returnFormat: 'json'
		},
		dataType: 'json',
		success: function(d, r, o){
			if (d.success) {
				grid.jqGrid('delRowData',rowId);
				dlg.dialog('close');
			}
			else {
				$('span#grid-dialog-error-message').html(d.message);
				$('div#grid-dialog-error').dialog('open');
			}
		}
	});
};

This will make an ajax call to remove the record. If the call is successful, then the row will be removed and the dialog closed. If it fails, then our new error dialog will open to tell us why. This tells us a little about how we need to form our remote method.


/**
 *	FUNCTION deleteEntry
 *	Used to remove entries from the system
 *
 *	@access remote
 *	@returnType struct
 *	@output false
 */
function deleteEntry(required string recId){
	LOCAL.retVal = {"success" = true, "message" = "", "data" = ""};

	// BEST PRACTICE: You'll want to verify that the user has the right to do this. Normally, that would go here.

	LOCAL.sql = "DELETE FROM tblblogentries
				 WHERE id = :recId";
	LOCAL.q = new Query(sql = LOCAL.sql);
	LOCAL.q.addParam(name = "recId", value = ARGUMENTS.recId, cfsqltype = "cf_sql_varchar");
	try {
		// You would uncomment the following line to actually remove records, and remove the throw statement
		// LOCAL.q.execute();
		throw (message = "Intentional Exception: You didn't really think I'd delete entries, did you?", type = "custom_err", errorCode = "ce1001");
	} catch (any excpt) {
		// In testing, and with the .execute() commented out above, comment out the next line to watch the grid remove a row
		LOCAL.retVal.success = false;
		LOCAL.retVal.message = excpt.message;
	}
	return LOCAL.retVal;
}

Yes, I didn't really delete anything. The code is there, but we're not killing entries today. This does give you the basic idea on how this all works though. Trying to delete an item in the grid now will show you a confirmation dialog. When you click on OK the ajax call is made. Right now, we intentionally throw an error. The success marker causes the error dialog to display with the message. If you comment out the first line of the catch, in this method, and re-run your test, you will see the confirmation dialog close, and the deleted record's row removed from the grid.

Old School Event Handler

Nowadays, it's standard practice to bind event handlers at runtime. It's even referred to as best practice. That said, there are issues with these new methods, particularly in making sure to unbind events from items you're removing from the DOM (or when they aren't needed at all). Binding events at runtime has runtime implications, and can cause memory leaks if not carefully controlled. For this reason, sometimes it just makes more sense to take it on the old school way. To demonstrate, let's take a different tact with how we handle our edit icons.


$.extend($.fn.fmatter, {
	actionFormatter: function(cellvalue, options, rowObject) {
		var retVal = "";
		retVal += "";
		return retVal;
	}
});

We put a standard anchor tag in there, with the onclick attribute calling a new method, editEntry, to which we pass the Entry ID (cellvalue). It's not elegant, but it works. To prove that, let's build out our editEntry method. We don't want to build a full Blog entry editor in this blog post, but we can create a quick page to load in a dialog via an ajax call.







	You must supply an ID of a record to edit!

You are editing #FORM.recId# This is just a simple template that takes a single POST variable (recId) and outputs it on the page. Now all you need is the ajax call to render this in a dialog box. var editEntry = function (recId) { $('

').dialog({
		title: 'Entry Editor',
		modal:true,
		width: $(window).width()*.6,
		height: $(window).height()*.6,
		create: function(){
			$('span.ui-icon-closethick').html("");
		},
		open: function(){
			var dlg = $(this);
			// Get the summary for the selected lesson
			$.ajax({
				url: '/edit.cfm',
				type: 'POST',
				data:{
					recId: recId
				},
				dataType:'script',
				success: function(d, r, o){
					dlg.html(d);
				}
			});
		},
		close:function(){
			$(this).html('').dialog('destroy');
			setTimeout('$("#recordEdit").remove();',100);
		},
		buttons:[{
			text:'Close',
			click:function(){
				$(this).dialog('close');
			}
		}]
	});
};

Here we create a div element on the fly. Pay special attention to our close config option, where we have to pull off some trickery to destroy the JQueryUI Dialog, and remove the div from the DOM. This is very important, because every time we click on an edit icon it is going to create and open a new dialog. In the open config option we make an ajax call to retrieve edit.cfm, passing it the record ID (recId) of the record we want to edit. We place the return of the request in the content of the JQueryUI Dialog, which we sized to take up 60% of the window size.

Wrap It Up

At this point I think we have a pretty good "Intro" to jqGrid. If there's some critical piece of info you think I have missed, please feel free to let me know through the Contact link below. Sample code for our application can be found below, from the Download link. As always, give me your feedback and questions.

ColdFusion JSON Serialization Changes

Tue, 28 Feb 2012 13:23:00 -0400

Because I've had to search on this three times in the last year, I thought it'd be a good idea to document this here, for myself and others. There were a number of important changes that can trip you up from version to version and, if you know which version the changes came in it can be beneficial. First is a change that affects JSON output, though it's not specifically about JSON. The ColdFusion 9.01 updater (the current base install, at the time of this post) addressed Bug 82980 by removing ColdFusion Debug Output from direct CFC requests. This was only important if you had debugging output turned on in the Administrator, but caused lots of issues when testing ajax applications in development environments that kept that setting on by default. Unfortunately, Adobe made some changes to JSON serialization that were not only unpopular, but technically dead wrong. They were trying to resolve the treatment of numbers in JSON serialization. Numbers were being converted to floats (1 became 1.0, 12 became 12.0, and so forth), so Adobe changed that by converting all numbers into strings (11 became "11", 17.24 became "17.24"). This was wrong too, as numbers weren't numeric anymore. At the time it raised quite a stir. They fixed this (Bug 83638) really quickly, releasingCumulative Hotfix 1 for ColdFusion 9.01, which fixed the issue of integers being converted to float and having them as numeric values (ie: [1,12,true,"this var",17.24]). Hurray! Except.... Enter the improperly titled Cumulative Hotfix 2 for ColdFusion 9.01. I say this because it really isn't 'cumulative'. Most things are straight, from a cumulative standpoint, with the seeming exception of the fix to JSON formatting. If you have not applied CHF 1 prior to installing CHF 2, then you will still have issues with JSON formatting. Overall, Hotfix 2 is great, fixing many bugs. The install is crazy/scary, and must be followed to the letter to prevent major issues, but it's worth it. As long as you did apply CHF 1 first, that is. Thankfully the hotfix/upgrade install process is much better in ColdFusion 10. Big thanks to Ray for helping me to realize that I wasn't crazy, and pointing me in the right directions for all of this info. Hopefully this helps someone down the line.

ColdFusion 10 Public Beta

Tue, 21 Feb 2012 16:14:00 -0400

Yes, everyone is talking about it. That's why I waited a day or two to post the info. Adobe has put ColdFusion 10 up on Adobe Labs, opening it's public beta cycle prior to full release. That's right folks, ColdFusion is finally hitting the double digits, and this release just continues to improve this great platform. There are some big changes. First are some overdue upgrades, like retiring the JRun JEE server, replacing it with Tomcat. Web Services support is upgraded, moving to more current form of Axis, and adding native support for creating RESTful web services. Exchange integration support is now upgraded to support MS Exchange 2010. Java and Solr integrations are improved as well. Finally, we get hotfix notifications and one touch upgrade directly within the administrator. I'm just getting started. ColdFusion has always been known for making the hard things easy, and it looks like ColdFusion 10 is no exception. Adobe's engineering team listened to customer feedback, giving us a lot of new toys to play with, including some great language enhancements. Highlights include important new array functions, app specific in-memory file system, MIME type checking on file uploads, an implicit CFC constructor, method chaining, and much, much more. Oh, and closures. Did I mention closures? Not sure how this one will play out, yet, but I look forward to seeing what people come up with. There's also websockets, media player changes, and an entirely new charting to work with. Just scratching the surface here. Yes, there is a lot of new stuff. And, to top it all off, Adobe placed ColdFusion Builder 2.01 in Beta at the same time. There are a lot of bug fixes in here, along with the CF 10 support and more. Did I mention both the server, and the IDE, have some HTML 5 goodness baked in as well? No? You'll just have to download them and start playing. This'll be fun.

MSOC Part 9: Application Event Handlers, The Rest

Mon, 13 Feb 2012 10:33:00 -0400

So far we've spent a lot of time going over the different Application Event Handlers that relate to specific scope events (application, session, and request). Some ask "How does this relate to MSOC?", to which I have to say "Because it's important to understand application flow and control." We'll be diving into the MSOC specific bits much more in the coming posts. For now, let's wrap this part of our MSOC talk with a brief discussion of our two final Application Event Handlers: onMissingTemplate and onError. The onMissingTemplate handler kind of does what it says. ALERT: Pay Attention To This Next Statement. This handler will be automatically invoked when a ColdFusion template or component is called but does not exist. Did you catch that? Yes, this only catches requests for templates/files that might be served by the ColdFusion application server. Setup a special mapping for CF to process .inc files? Yes, if the file doesn't exist, then this handler should catch that. A .html page was requested? Unless the server is mapped for CF to handle those requests, then no, your onMissingTemplate method will not catch that error. You get the picture yet? The basic layout of the onMissingTemplate method is like this:


/**
 *    FUNCTION onMissingTemplate
 *    Runs when a (CF) template is called that does not exist
 *    @access public
 *    @returnType boolean
 *    @output false
 */
function onMissingTemplate(required string targetpage) {
	// something goes here
	return true;
}

As you can see, this method is automatically passed one argument, the page being requested. Here's another catch for you, this will only catch requests for ColdFusion processed files in the same directory as your Application.cfc. So a request for mysite.com/nothere.cfm will be caught, but a request for mysite.com/some/random/path/to/nothere.cfm will not (This surprised me too, and I had to test it over and over again to make sure I wasn't missing something). Weird, huh? Well, that's ok. To be honest, the onMissingTemplate handler is better served by those who don't have access to change their own webserver. The better option is to setup a 404 handler in their web server configuration. In the Apache Web Server you can look for the ErrorDocument line, in your httpd.conf file, and set it to a specific template that you setup for error handling.


ErrorDocument 404 /errors/404.cfm

Speaking of error handling... You do do error handling, right? You wouldn't just show end users hard errors, would you? Of course not. You've got try/catch, throw and rethrow all over the place, right? I've never seen an app that was made to handle every possible scenario for what a user could do. Nope, never. And, I've never seen (or written) a perfect app. It's a mythical creature, like the unicorn, or honest politicians.Fortunately, Coldfusion gives us multiple levels of error handling capability, from the block level of try/catch to the server level error handler applied through the CF Administrator. The onError method allows us to apply an application level error handler, that catches anything not previously coded for (through try/catch). It can display (if desired) any error thrown from onApplicationStart through onRequestEnd, including errors thrown through the process of the request itself, but can not display content for errors thrown in the onSessionEnd or onApplicationEnd. The onError event handler is formed like this:


/**
 *    FUNCTION onError
 *    This is an application wide error handler. Best practice would be to
 *    write process specific error handling, but this method will help
 *    you trap unexpected errors for custom notification and process
 *    @access public
 *    @returnType void
 *    @output true
 */
function onError(required  exception, required string eventname) {
	if(StructKeyExists(APPLICATION.cfc, "errorHandler")){
		// If the error is thrown in onApplicationEnd or onSessionEnd, the error processor
		// will still run, but nothing will be displayed to the user
		WriteOutput(APPLICATION.cfc.errorHandler.process(argumentCollection: ARGUMENTS));
	} else {
		WriteDump(var = ARGUMENTS.exception, abort = true);
	}
}

The onError method is automatically passed to argments, exception (think CFCATCH) and eventname (the application event that the error was thrown in). In my example, I've used an application wide error handler to process the error, and displayed it's output (if not thrown from onSessionEnd or onApplicationEnd). My handler could do any number of things, from logging to emailing the site admin to generating error code specific text response for output, to all of the above and more. OK, that's it for the Application Event Handlers. Let's be honest, I really only glossed over them and they still took up a lot of posts. That said, it's time to get back to the core MSOC talk, even as important as understanding all of this might have been. We have things to talk about, like directory structures, shared and specific asset handling, extended application models, and more. Give me your feedback: What are your most pressing questions on running Many Sites with One Codebase?

The Joys of Developing for Internet Explorer

Thu, 09 Feb 2012 10:51:00 -0400

Note: Follow the madness here to it's conclusion, to discover yet another "Really? I didn't know that..." IE moment, that may save you heartache and pain. Is the sarcasm evident in my title? It should be. While Internet Explorer may have been "groundbreaking" when it was released, it has ever been the bain of the web developer's existance. Why, you ask? Because it refuses to adhere to standards, and just operates differently than everyone else. To be fair, Internet Explorer 10 is in the works, and supposedly closes the gap a fair amount (and even wins out in some of the html 5/css 3 support), and 7, 8 and 9 did progressively improve (if slowly). Unfortunately the fact remains that some companies/organizations/governments are stuck on Internet Explorer 6, or maybe even 7. If you are a web developer, and write cross-browser web applications, chances are Internet Explorer has hit you more than once in your career. It's the dumbest things that hit you too. The most obscure "WTF!?!" moments that drive you crazy. That is a daily experience for me now.I've been a server-side developer for over two decades now, but in the last several years I have dealt more and more with client-side development. Why? Well, I started with HTML, and got into Javascript when it was introduced, and then got into server-side programming, so I already had some roots. With all of the work that I've done, working with Ext JS and JQueryUI, my current position threw me to the wolves to work on standardizing our application's interface development. Fun, right? It is (a lot), but the client is one of those mixed environments that is entrenched in IE, typically at 7 or 8. They can't upgrade, so it's our responsibility to make sure that our interfaces can render and function properly in these older, buggy, non-compliant POS browsers. (Note the frustration here.) Let me layout one of those typical, what-the-hell-is-goin-on-here, Internet Explorer moments for you.

The Application

Like most good development companies, we've made the shift to more modular, reusable code. It's a legacy codebase, serving the client well for years, and goes through constant review, revision, update and improvement. Many of us (developers in general) would love to just scrap our current codebase and start from scratch, but that's not really practical in most situations. What happens is you begin to slowly move away from The Big Ball of Mud design pattern, by replacing small pieces of your application as updates/changes are required, with more modern code. Rather than one huge application, what you end up with are small, mini-applications, that can be plugged in anywhere within an application. Each of these 'applications' then have their own supporting files: html, javascript, and css. Welcome to the fun.

Collision Control

At this point, the issue isn't necessarily browser specific, but rather logistics. When you start adding in multiple mini-applications, things can (and will) butt heads. You will have issues with race conditions, as multiple scripts use variables of the same names, have functions with the same name that override each other due to execution order, id's repeated in a page (IE hates this a lot), and multiple event handlers accidentally added to the same DOM controls because selectors aren't specific enough. You tighten up your DOM selectors, come up with some variable and DOM naming standards, and learn about namespacing. Each a slow, progressive step towards sanity. Then there's the issue of accidentally loading the same files multiple times. Our application uses JQueryUI, cfUniform, TinyMCE, and jqGrid extensively. Each of these mini-apps loads it's own support files. Suddenly, you're faced with coming up with systems of control, to ensure that you aren't loading scripts and CSS multiple times.

Back to the Browser

There's a ton of debate out there about loading multiple scripts and css files. Loading multiple files has performance issues for initial download, but maintenance is far easier with these bit modularized like your display and model. In many applications things aren't complex enough for multiple files to truly be an issue. But, when you are writing a complex application, like scheduling software, customer resource management, content management, etc., the number of 'modules' you might include in a page can grow and grow and grow, especially if your interface makes heavy use of ajax. This is where Internet Explorer can actually force your hand into creating a better experience, unintentionally and generally when it's inconvenient. What happens when you load up your app, and notice that your forms (cfUniform) don't appear to be styled, and your highly configured and customized WYSIWYG editor (TinyMCE) appears to be half loaded? But, only in IE? Well, you start troubleshooting. Maybe cfUniform didn't load? No, the stylesheet is what's important here, and Developer Tools say it's there. The editor? Did it initialize? Well, the style and font dropdown info is there, even if they aren't in dropdowns. What's going on? You start removing script files, one at a time, to make sure there isn't a conflict. But wait, it works in everything but IE? Well, you try it anyway. You step debug things, trying to see a break. And you Google (and Google, and Google). These are my days. And, after three days of this single issue, you finally stumble on an obscure post (page 15 of a Google search). Please, tell me you're kidding me... Yes, Internet Explorer (at least 6 through 9) only apply up to 31 stylesheets and/or style blocks. So, when your page (1 specific stylesheet) uses JQueryUI (1 plugin stylesheet), and a menu (3 plugin stylesheets), and a nav widget (1 plugin stylesheet), plus cfUniform (3 plugin + 2 override stylesheets), and jqGrid (1 plugin stylesheet), and TinyMCE (multiple dynamically loaded stylesheets, depending on plugin configuration), and... You get the picture. Oh, and did I mention that some of those stylesheets were 10 line, IE only stylesheets to hack IE's different handling of CSS? Yeah, salt in the wound. To test this, I went looking for something that might be loaded by default, but wasn't needed on this page of my app (or at least, not for what I was testing). I found a few stylesheets I could disable in local development, and reloaded my page. Imagine my surprise when everything rendered as it should. I had to toggle it back and forth a few times just to verify. I think I spewed profanity for several minutes.

What To Do?

OK, so we had already been looking to form a strategy for combining files and minification. Notice I said "looking to". That is to say, we wanted one, and knew we needed one, and even done a little research, but we hadn't solidified one. If we had, we'd already have been working on it. Now, under the gun, with deadlines looming, we have to put it in gear. What to do? Well, in this case, improvise. Time not being on our side, we have to do this manually. Identify multiple CSS files that are used either constantly (every single page load) or extensively (85%+ page view). Combine and compress, and remove single link references from the code. Now, I didn't do this completely manually. I did write a script where I could define which files, in what order, and have the script build the file, while correcting internal url references so that image paths wouldn't break. Then I used an online compressor to minify the file. After all of this work (took the good part of an afternoon), I reload my pages and all of them function as intended, in all browsers. And, compressing 10 files down to 1, I now get my form styling, and my editor displays correctly, etc. OK, is this the ideal solution? No, we'll still develop a comprehensive (and automated) strategy. But, this gets us over the deadline hump, and proves that the multiple stylesheets were the issue, and that this type of action can correct the issue. Proof of concept. You have to start somewhere I guess. But, how much easier web development would be if we didn't have to support Microsoft's mistakes.

Intro to jqGrid Part 5: Search

Fri, 13 Jan 2012 14:20:00 -0400

At this point we've created a basic grid, filled it with data, refined the display of our columns and added event handlers to handle multiselect options. We've used custom cell formatters, used a custom datatype function, and even added and populated a toolbar in the process. Now let's start looking at some things that aren't necessarily jqGrid specific, but incorporate them for use in our grid. How about a search? jqGrid includes some things for doing data search, that will automatically build modal windows and stuff. But sometimes you want to format things your own way, or incorporate jqGrid for use within an existing interface. One of the advantages for us, using the datatype function, is that we can preprocess our postdata prior to the ajax request.First, let's add a simple form to our example:

index.html - Simple Search



	
		Title

		

		
			From

			
		
		
			To

			
		


        To search for entries on a specific date, use the To field only.

...

Looking at our demo grid, I only truly have two fields to search on: Title and Release Date. So, we provide for searching against our Title, and for searching within a date range. To top off our simple search form, let's turn those from and to fields into JQueryUI DatePicker fields.

jqGridDemo.js - DatePicker initialization


$('input.addDatePicker').datepicker({
	showOn: 'button',
	buttonImage: '/resources/images/icons/calendar.png',
	buttonImageOnly: true,
	dateFormat: 'mm/dd/yy'
});

This adds a DatePicker element to any input with the addDatePicker class. At some point you'll want to add some range validation (client and server side). For the moment the action attribute is blank. That's OK, for us, as we're going to provide a custom submit handler in our script:

jqGridDemo.js - Search Form Submit Handler


$('form#searchForm').submit(function(ev){
	ev.preventDefault();
	// something will go here
	return false;
});

Next, we're going to think about how we're using our 'search'. Basically, this is a filter we're going to apply through our query: a list of fields and values to narrow our result set by set criteria. We can param all our fields in our remote method (and should), so to avoid confusion we're going to 'scrub' our form prior to making our ajax request. Basically, we don't need anything that's empty, or with a value of '0'.

jqGridDemo.js - scrubSearch


var scrubSearch = function(){
	var frm = $('form#searchForm').serializeJSON();
	for(var i in frm){
		var val = frm[i];
		// if value has no length, remove the key
		if(val.length === 0){
			delete frm[i];
		}
		if(!isNaN(val-0) && parseInt(val) === 0){
			delete frm[i];
		}
	}
	return frm;
};

"Wait a minute, Cutter. JQuery doesn't have a serializeJSON method." No, it doesn't. JQuery has serializeArray, for pulling form values out, but doesn't have a nice method for pulling that into object notation, which is easier to work with. Arjen Oosterkamp posted this little plugin, in the comments of the serializeArray documentation. While good, the one fault I had was that it didn't turn repeating field name (like checkbox usage) values into a list, which is standard behavior on a form submit, so I modified his work to accommodate.

jqGridDemo.js - serializeJSON


// Before your document ready statement
(function( $ ){
	$.fn.serializeJSON=function() {
		var json = {};
		jQuery.map($(this).serializeArray(), function(n, i){
			(json[n['name']] === undefined) ? json[n['name']] = n['value'] : json[n['name']] += ',' + n['value'];
		});
		return json;
	};
})( jQuery );

By applying this method to our form, we get a nice object that we can then loop through for value comparisons. We return the object after we've removed any keys with empty or 0 values. Since form values are strings by default, we have to work a little magic for finding numeric values (JQuery 1.7 includes the new $.isNumeric() method, but we aren't using 1.7 in our examples). Now, on first look you might think we're going to call this from within our form's submit handler, but that's not the case. Our example is using a basic html page, but you might be using a dynamic page where you may pre-populate your form fields with data passed in the initial page request, to pre-filter grid results for some reason or another. For this reason, you would want to have this method called immediately on page load, and on any additional request as well. For this, we'll go back to our populateGrid method.

jqGridDemo.js - populateGrid


var populateGrid = function (postdata) {
		$.ajax({
			...
			data: $.extend(true, {}, postdata, {search: $.toJSON(scrubSearch())}),
			...
	};

We used JQuery's $.extend() method to create a new data object, combining our postdata with a new object including search. JQuery (I wish I knew why) does not include anything for converting an object to a JSON string (only the reverse, with $.parseJSON()), so we must include another plugin library for making this possible. Luckily there's the jquery-json project on Google Code, which is heavily influenced by work done at JSON.org. All we need, to access the $.toJSON() method, is to add the file to our html head.

index.html - header script includes


...



...

By handling this, in this fashion, the form's contents are taken into account on every grid load request. This means that paging and sorting requests will have the filter criteria, until such time as the form is cleared. It also means that our form's submit handler becomes that much simpler.

jqGridDemo.js - Search Handler figure 2


$('form#searchForm').submit(function(ev){
	ev.preventDefault();
	grid.trigger('reloadGrid');
	return false;
});

This is great! When you submit the form the grid's contents are now filtered by your search criteria. The only thing you have to do now is have a way to 'clear' your search criteria. You can easily do this by adding a reset button to the form, and binding to the reset event.

index.html - Search Form Buttons


...
 


...

jqGridDemo.js - Search Form Reset Handler


$('form#searchForm').bind('reset',function(ev){
	setTimeout("$('#gridTest').trigger('reloadGrid');",1);
});

Our binding needs us to give the browser a moment to 'clear' the field values, prior to us triggering a grid reload. Since setTimeout will have no concept of scope, we cannot use our grid variable here. This all takes care of things, from an interface perspective, but we still need to handle these filter requests at the server. We're going to first change our remote method signature to accept our new search parameter, which is now included on every request as a JSON string.

Entries.cfc - getEntries Method Signature


function GetEntries(numeric pageIndex = 1, numeric pageSize = 50, string sortCol = "ID", string sortDir = "desc", string search = "") {

We can then test the argument for value, and param all our search fields

Entries.cfc - getEntries figure 2


function GetEntries(numeric pageIndex = 1, numeric pageSize = 50, string sortCol = "ID", string sortDir = "desc", string search = "") {
	...
	if(Len(ARGUMENTS.search) AND IsJSON(ARGUMENTS.search)){
		ARGUMENTS.search = DeserializeJSON(ARGUMENTS.search);
	} else {
		ARGUMENTS.search = {};
	}

	param name="ARGUMENTS.search.title" default="";
	param name="ARGUMENTS.search.from" default="";
	param name="ARGUMENTS.search.to" default="";
	...
}

This now gives us some things to key off of for building our 'filtered' query. Parts of this are fairly simple: If the field exists, add the filter to the query, and the param to the query definition. The title is a good example.

Entries.cfc - getEntries figure 3


...
// Main data query
LOCAL.sql = "SELECT	SQL_CALC_FOUND_ROWS id,
				title,
				posted,
				views
			FROM	tblblogentries
			WHERE 0 = 0
			 ";
if(Len(ARGUMENTS.search.title)){
	LOCAL.sql &= " AND title LIKE :title
	 ";
}
LOCAL.sql &= "ORDER BY #LOCAL.orderby#
			 LIMIT	:start,:numRec";
LOCAL.q = new Query(sql = LOCAL.sql);
LOCAL.q.addParam(name = "start", value = (ARGUMENTS.pageIndex-1) * ARGUMENTS.pageSize, cfsqltype = "cf_sql_integer");
LOCAL.q.addParam(name = "numRec", value = ARGUMENTS.pageSize, cfsqltype = "cf_sql_integer");
if(Len(ARGUMENTS.search.title)){
	LOCAL.q.addParam(name = "title", value = "%#ARGUMENTS.search.title#%", cfsqltype = "cf_sql_varchar");
}
...

The Release Date stuff is a bit more tricky, as you have to think about how dates are treated. First, you may not need to worry about working with the from and to fields at all. We can do a little pre-check right after our param statements.

Entries.cfc - getEntries figure 4


...
LOCAL.hasFrom = Len(ARGUMENTS.search.from) AND IsDate(ARGUMENTS.search.from);
LOCAL.hasTo = Len(ARGUMENTS.search.to) AND IsDate(ARGUMENTS.search.to);
...

This allows us to verify that a value was passed, and that the string is a valid date value. Next we have to consider our logic. We stated on our form that the user can search for a specific day by only filling the To field. This means that, at the least, we need to have value in the To field to filter Release Date.

Entries.cfc - getEntries figure 5


// Main data query
LOCAL.sql = "SELECT	SQL_CALC_FOUND_ROWS id,
				title,
				posted,
				views
			FROM	tblblogentries
			WHERE 0 = 0
			 ";
if(Len(ARGUMENTS.search.title)){
	LOCAL.sql &= " AND title LIKE :title
	 ";
}
if(LOCAL.hasTo){
	LOCAL.sql &= "AND posted BETWEEN :from
	 AND :to
	 ";
}

What you see here is that we've added an AND BETWEEN clause to the query if to is passed in. But, why BETWEEN? The posted field, in our database, is a DATETIME datatype. So, for instance, you searched for entries posted on 08/01/2011 you would actually need to search the database for anything posted BETWEEN 00:00:00 and 23:59:59 of 08/01/2011. We need to handle either a range, or a single date. The basic sql is the same (AND BETWEEN statement), so what we have to concern ourselves with is how we create our from and to sql paramaters.

Entries.cfc - getEntries figure 6


if(LOCAL.hasFrom AND LOCAL.hasTo){
	LOCAL.q.addParam(name = "from", value = CreateODBCDateTime(ARGUMENTS.search.from), cfsqltype = "cf_sql_timestamp");
	LOCAL.q.addParam(name = "to", value = CreateODBCDateTime(ARGUMENTS.search.to & "23:59:59"), cfsqltype = "cf_sql_timestamp");
} else if (!LOCAL.hasFrom AND LOCAL.hasTo){
	LOCAL.q.addParam(name = "from", value = CreateODBCDateTime(ARGUMENTS.search.to), cfsqltype = "cf_sql_timestamp");
	LOCAL.q.addParam(name = "to", value = CreateODBCDateTime(ARGUMENTS.search.to & "23:59:59"), cfsqltype = "cf_sql_timestamp");
}

CreateODBCDateTime will create a sql date object format that the JDBC connector will properly convert for it's db and datatype. Thing is, when given a date, without time, then the date defaults to {ts 'yyyy-mm-dd 00:00:00'}. For this reason, our to value must add on the time to the last second of that day, to properly search any entries through that day. The says if a from and to were passed, then filter between the beginning of from and the end of to, else if only to was passed, then filter from the beginning of to to the end of to. The last thing to notice in these blocks of code is the concatenation of the LOCAL.sql string that builds our query. You may notice, in looking at these code samples, that there appears to be an additional line added to each. This is to accomodate a bug in ColdFusion's scripted sql parser, in dealing with whitespace when using sql parameters. If you look at this code with whitespace characters being shown, you will see the additional spaces required.

And that's it! Now, when you submit search criteria, your grid will be filtered on the information you've sent, and retain your criteria across paging criteria until you clear the search form.

That's it for now. In our next post we'll finally get around to those icons in the Action column. Example code for this post may be found in the Download link below.