ColdFusion 10 File Uploads and MIME Types

Quick Note: On one project that I'm on, we're in the process of moving from ColdFusion 9 to 10. During some regression testing, one of the testers began receiving errors on file uploads. When looking at the code, I saw this:

view plain print about
1fileUploadResult = fileUpload(arguments.tierCSSDirectory, "header_background_image", "image/*", "overwrite");

Now, I remembered that there were changes to MIME type checking in CF 10, but I wasn't entirely sure. This code, written by one of our developer's several months ago, would allow any "image" MIME type. What we discovered is that we couldn't do this kind of wildcard mapping under CF 10, that we now had to list out each accepted MIME type as a comma delimited list.

Just thought I'd share.

ColdFusion Roadmap

Shilpi, the 'Security Czar' of Adobe's ColdFusion Engineering team, recently posted about the Roadmap for future ColdFusion Server Devlopment, post CF 10. This links to an abbreviated set of slides, presented in a PDF document, that layout the focus on next two versions of CF.

There are some things you might easily miss, in reading over the slides. First thing I picked out was in "Splendor"'s Other focus areas section, you see Pluggable Framework mentioned. Hmmmmmm...

Mobile, Social, HTML 5 and the Cloud are big topics here, but I also noted that "Dazzle" has Customizable Enterprise Video Portal mentioned. Interesting...

Another one that's easy to miss? How about the timeline? According to the last slide, "Splendor" looks like a 2013 release, while "Dazzle" is slated for 2014. That's a much more progressive release cycle than we're used to seeing.

The biggest, immediate take away here, is the fact that Adobe is obviously investing time and resources in to continuing to produce and progress the ColdFusion platform. That's a win.

ColdFusion 10 is Now Available

Adobe has released ColdFusion 10. The engineering team has done a fantastic job putting this release together. Here's a short list of some of what's new:

  • Improved/Updated
    • caching
    • ORM
    • webservices
    • scheduler
    • hotfix management
    • security
    • MS Exchange integration
    • security
    • Solr
    • charting (now html 5)
    • Java integration
    • Flex and AIR lazy loading
    • VFS
    • more!
  • html 5 support
  • websockets
  • RESTful services
  • greater cfscript parity
  • geo-tagging
  • CFC auto constructors
  • method chaining
  • closures
  • Tomcat (no more JRun)
  • more!
    • Wow! A lot of changes and additions. The team has been in overdrive, and it all looks spectacular. Download the free Developer's Edition and start playing.

      As a bonus, they also release an update to ColdFusion Builder as well!

ColdFusion Security Hotfix and Big Forms

The other day, Adobe released a new Security Hotfix for it's ColdFusion server. There were a number of things addressed in the hotfix, to help protect against Denial of Service attack using a hash algorithm collision. (My wife would say I sound like Charlie Brown's teacher right about now.) Ok, the important thing is you need to update your server.

Now for the fun part. We loaded the fix to our testing servers to run our app around the block prior to pushing this up to production. And, it's a good thing we did. We're preparing for a large deployment, and testing is pretty heavy right now. First thing in is that a form would no longer submit, throwing a 500 error every time. I didn't show me a 500 error, just a blank page. I had to look at Firebug to see the error code response. Now, if you've ever encountered a 500 error from the server then you know they don't typically tell you much. I reproduced the error locally and then went looking through the log files on the server.

In a multi-server configuration there are two core areas to look at log files. The first are the basic JRun logs. On a Windows systems, these files are located in the C:\JRun4\logs folder. Here you will typically find a {instance}-out.log file, and a {instance}-event.log file, for each ColdFusion instance you have configured. Right out of the gate you have admin-event and admin-out logs for the JRun administrator, and cfusion-event and cfusion-out for the default ColdFusion instance. I checked both files for my instance, and saw there weren't any items to tell me about the 500 error, so I then went looking at the ColdFusion logs.

Each ColdFusion instance has it's own set of log files, that you can see in the logs viewer in the ColdFusion Administrator. That said, the ColdFusion Administrator is not really the best place to go through these files, especially when you're really having issues. At this point, you just want to open them yourself. First, you have to find them. You do this through your instance, C:\JRun4\servers\{instance}\cfusion.ear\cfusion.war\WEB-INF\cfusion\logs. You'll probably find multiple log files here, from the application and eventgateway and mail logs, to individual logs from cflog calls.

Our issue, with submitting our form, was answered by the JRun -event logs, which gave me a few error messages saying something like this:

view plain print about
103/27 07:52:00 error ROOT CAUSE:
2coldfusion.filter.FormScope$PostParametersLimitExceededException: POST parameters exceeds the maximum limit specified in the server.
3    at coldfusion.filter.FormScope.parseQueryString(FormScope.java:397)
4    at coldfusion.filter.FormScope.parsePostData(FormScope.java:346)
5    at coldfusion.filter.FormScope.fillForm(FormScope.java:296)
6    at coldfusion.filter.FusionContext.SymTab_initForRequest(FusionContext.java:377)
7    at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:33)
8    at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
9    at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)
10    at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:126)
11    at coldfusion.CfmServlet.service(CfmServlet.java:200)
12    at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
13    at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
14    at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
15    at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
16    at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
17    at jrun.servlet.FilterChain.service(FilterChain.java:101)
18    at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
19    at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
20    at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
21    at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
22    at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
23    at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
24    at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)

"Dude! What is that!?!" Well, luckily I had installed my security hotfix, locally, just that morning, so I remember reading over the instructions. In it's notes it had stated the following:

  1. Customers who want to change postParameterLimit, go to {ColdFusion-Home}/lib for Server installation or {ColdFusion-Home}/WEB-INF/cfusion/lib for Multiserver or J2EE installation. Open file neo-runtime.xml, after line
    view plain print about
    1"<var name='postSizeLimit'><number>100.0</number></var>"
    add the below line and you can change 100 with desired number.
    view plain print about
    1"<var name='postParametersLimit'><number>100.0</number></var>"

Just a heads up, that neo-runtime.xml file is minified, so you'll want to Find "postSizeLimit" to get that statement in the right place. We tried that postParametersLimit value (100) and found that our form had more than that (many were hidden, but that's another post all together), so we adjusted the number to 200. After restarting the instance again, we tested the form once more with complete success.

Hopefully this will help someone else avoid this issue. It's important to remember that Adobe does try to document these types of situations with hotfixes, so when you run into issues they should be your first source of troubleshooting information.

ColdFusion JSON Serialization Changes

Because I've had to search on this three times in the last year, I thought it'd be a good idea to document this here, for myself and others. There were a number of important changes that can trip you up from version to version and, if you know which version the changes came in it can be beneficial.

First is a change that affects JSON output, though it's not specifically about JSON. The ColdFusion 9.01 updater (the current base install, at the time of this post) addressed Bug 82980 by removing ColdFusion Debug Output from direct CFC requests. This was only important if you had debugging output turned on in the Administrator, but caused lots of issues when testing ajax applications in development environments that kept that setting on by default.

Unfortunately, Adobe made some changes to JSON serialization that were not only unpopular, but technically dead wrong. They were trying to resolve the treatment of numbers in JSON serialization. Numbers were being converted to floats (1 became 1.0, 12 became 12.0, and so forth), so Adobe changed that by converting all numbers into strings (11 became "11", 17.24 became "17.24"). This was wrong too, as numbers weren't numeric anymore. At the time it raised quite a stir.

They fixed this (Bug 83638) really quickly, releasingCumulative Hotfix 1 for ColdFusion 9.01, which fixed the issue of integers being converted to float and having them as numeric values (ie: [1,12,true,"this var",17.24]). Hurray! Except....

Enter the improperly titled Cumulative Hotfix 2 for ColdFusion 9.01. I say this because it really isn't 'cumulative'. Most things are straight, from a cumulative standpoint, with the seeming exception of the fix to JSON formatting. If you have not applied CHF 1 prior to installing CHF 2, then you will still have issues with JSON formatting.

Overall, Hotfix 2 is great, fixing many bugs. The install is crazy/scary, and must be followed to the letter to prevent major issues, but it's worth it. As long as you did apply CHF 1 first, that is. Thankfully the hotfix/upgrade install process is much better in ColdFusion 10.

Big thanks to Ray for helping me to realize that I wasn't crazy, and pointing me in the right directions for all of this info. Hopefully this helps someone down the line.

ColdFusion 10 Public Beta

Yes, everyone is talking about it. That's why I waited a day or two to post the info. Adobe has put ColdFusion 10 up on Adobe Labs, opening it's public beta cycle prior to full release. That's right folks, ColdFusion is finally hitting the double digits, and this release just continues to improve this great platform.

There are some big changes. First are some overdue upgrades, like retiring the JRun JEE server, replacing it with Tomcat. Web Services support is upgraded, moving to more current form of Axis, and adding native support for creating RESTful web services. Exchange integration support is now upgraded to support MS Exchange 2010. Java and Solr integrations are improved as well. Finally, we get hotfix notifications and one touch upgrade directly within the administrator. I'm just getting started.

ColdFusion has always been known for making the hard things easy, and it looks like ColdFusion 10 is no exception. Adobe's engineering team listened to customer feedback, giving us a lot of new toys to play with, including some great language enhancements. Highlights include important new array functions, app specific in-memory file system, MIME type checking on file uploads, an implicit CFC constructor, method chaining, and much, much more.

Oh, and closures. Did I mention closures? Not sure how this one will play out, yet, but I look forward to seeing what people come up with. There's also websockets, media player changes, and an entirely new charting to work with. Just scratching the surface here.

Yes, there is a lot of new stuff. And, to top it all off, Adobe placed ColdFusion Builder 2.01 in Beta at the same time. There are a lot of bug fixes in here, along with the CF 10 support and more. Did I mention both the server, and the IDE, have some HTML 5 goodness baked in as well? No? You'll just have to download them and start playing. This'll be fun.

2011 In Review, and the View for 2012

My, how time flies when you're having fun! It seems like only yesterday that I was welcoming in 2011, and now we're here a year later. So many things have happened in the last year, and rereading that post I see that I missed some things I should've done, but let's take a look in retrospect.

I wrote 27 blog posts in 2011. This is nothing, compared to guys like Ray Camden or Ben Nadel, but for me it was quite a bit, especially when you consider that between March and August I released only one post. Very early in the year, I began a series on creatingmany sites with one codebase. In the process, the series has evolved to contain a fairly detailed primer in ColdFusion application architecture (because of it's importance to this process), has currently spanned 8 separate posts, and was even referenced by Sean Corfield in his great presentations on the same topic. 2012 will see the completion of that CF app discussion, and gradually move it back to the MSOC topic itself, as there is still a ton to talk about there, and a lot of interest in the topic. I also began a series on the jqGrid JQuery plugin. jqGrid is another Data Grid visualization tool (I have now written about three, including Ext JS and DataTables), and is a clear choice for those who must use JQuery. (To be fair, JQueryUI is working on a grid component, but they are still behind the curve, and way behind Sencha.) Finally, one common thread seen in the majority of my posts, is how much I've embraced cfscript. I wrote a lot of things, on a variety of topics, but most of my code examples were pure scripted examples.

Now let's talk about some other departures from the norm for Cutter.

You did not see a lot of content around Ext JS. In fact, I stopped writing Ext JS books. This is not, in any way, a reflection on my feelings for Ext JS. I still believe that Sencha has built one of the best client-side libraries for web application development. In evaluating the overall ROI, I realized that I was writing more for the community than the money, and that my reach was greater through my blog, while giving me flexibility on when and what I deliver from a content standpoint. That said, I didn't have a single project this year that used Ext JS, so had very little time to experiment and write about it. This year, I'm going to expand on a personal project, and get back to some great Ext JS content for my readers.

You, also, did not see me speak at any conferences this past year. Nor at any user group meetings. This wasn't because I didn't want to, but because of some more personal reasons. I'm not going to go in depth here, other than to say that I've had some long standing health issues that required me to have some surgery done on my mouth. (Mark Drew is making a joke right now...) Aside from the fact that this has been very costly (chewing up any conference/travel budget), it also meant that my speech has been affected for a good part of the year. Thankfully this experience is (mostly) over now, and I hope to get back to presenting sometime this year. Any user group looking for a speaker this year, please contact me through the Contact link on this blog.

One group I am hoping to speak to this year is the Northeast Florida CFUG. I have to call Mike back, but he's looking to get things kicked off again, and I want to help it be successful. If you're in or around the Jacksonville area, make sure to keep an eye on the site for upcoming events.

One other thing I'm looking to do is to migrate all of my projects into GitHub. I've been using Git at work, and I am loving it, and I think combining GitHub with RIAForge is a great way to promote the terrific technologies we work with every day. I will make the time, I promise.

This comes to the final discussion of this post, Adobe. I again had the pleasure of being an Adobe Community Professional this past year. Due to my health issues, I didn't get to do everything I would've wanted to this year, but I've tried to be a good supporter. There are some fabulous things coming in ColdFusion Zeus and, by extension, to ColdFusion Builder as well. There has been a lot of hub-bub over Adobe's communications flubs regarding Flash, mobile, and Flex. I've avoided much of the discussion, other than to say "be patient and watch". Flash isn't going away, and neither is Flex. HTML 5 is a beautiful thing, if you aren't developing desktop browser applications (i.e. You're only writing for mobile/tablet development). There, that is my whole contribution to that discussion. Give it a rest.

2012 will be a fantastic year. Set yourself some clear, definable goals. Break them down, step by step, and write the steps down on paper. Each successive step, print out in large letters and place it somewhere where you will see it each and every day. Set yourself up to succeed, and you will. Have a great year, everyone, and I can't wait to hear what you have planned for 2012.

A Scripted Query Param & Whitespace Gotcha

I discovered this one a while back, but forgot to write a post on it. Did you realize that formatting queries could affect the execution of scripted queries? Consider the following function:

view plain print about
1/**
2 *    FUNCTION login
3 *    A function to validate a user login, and return a struct of user details
4 *
5 *    @access public
6 *    @returnType struct
7 *    @output true
8 */

9function login(required struct formScope) {
10    var retVal = {"success"=true,"message"="","data"=""};
11    var sql = "SELECT     u.userID,
12                        u.username,
13                        u.password,
14                        u.dateCreated,
15                        u.lastUpdated
16                FROM    users u
17                WHERE     u.username = :username
18                AND        u.password = :password
19                AND        u.isActive = 1";
20    var q = new Query(datasource = VARIABLES.instance.dsn,sql = sql);
21    q.addParam(name = "username", value = ARGUMENTS.formScope.username, cfsqltype = "cf_sql_varchar");
22    q.addParam(name = "password", value = ARGUMENTS.formScope.password, cfsqltype = "cf_sql_varchar");
23
24    try {
25        retVal.data = LOCAL.q.execute().getResult();
26        // Check for no recordCount, and throw a 'no records' exception
27        if(!retVal.data.recordCount){
28            throw(type="MH-Custom",errorCode="001",message="The user " & ARGUMENTS.formScope.username & " could not be authenticated. Please check your credentials and try again.");
29        }
30    } catch (any excpt) {
31        retVal.success = false;
32        if(excpt.type eq "MH-Custom"){
33            retVal.message = excpt.message;
34        } else {
35            // TODO: Add admin notification in here somewhere
36            retVal.message = "There was a problem executing this request, and our administrators have been notified";
37            WriteDump(var=VARIABLES.instance,label="instance");
38        }
39        if(StructKeyExists(excpt,"errorCode") AND Len(excpt.errorCode)){
40         retVal["errorCode"] = excpt.errorCode;
41        }
42    }
43    return retVal;
44}

It's a pretty basic function, with a query to check submitted form fields against the database. Right? So, why would it error? "Error? What error?" Yes, it errors. Here's the code for a basic call, along with a dump to output that to the page:

view plain print about
1<cfscript>
2    REQUEST.testObj = CreateObject("component","com.multihome.core.Security").init(DSN='multihome');
3    REQUEST.test = REQUEST.testObj.login({username='admin',password='admin'});
4    WriteDump(var=REQUEST.test);
5
</cfscript>

Dumping that result shows you the error coming through:

CFDump 1

So, to get at the root of this I had to comment out all of my try/catch work:

CFDump 2

Whoops! Forgot my onError handler. OK, I'll comment that out. Here we go! Now we get to the meat of it (the dump was the same, but sometimes you just want to see the raw error):

view plain print about
1Error Executing Database Query
2
3Parameter 'username AND u.password' not found in the list of parameters specified
4
5SQL: SELECT u.userID, u.username, u.password, u.dateCreated, u.lastUpdated FROM users u WHERE u.username = :username AND u.password = :password AND u.isActive = 1
6
7The error occurred in C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\WEB-INF\cfusion\CustomTags\com\adobe\coldfusion\query.cfc: line 108
8Called from C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\WEB-INF\cfusion\CustomTags\com\adobe\coldfusion\query.cfc: line 137
9Called from C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\WEB-INF\cfusion\CustomTags\com\adobe\coldfusion\query.cfc: line 472
10Called from C:\JRun4\servers\cfusion\cfusion-ear\cfusion-war\WEB-INF\cfusion\CustomTags\com\adobe\coldfusion\query.cfc: line 605
11Called from C:\Inetpub\com\multihome\core\Security.cfc: line 68
12Called from C:\Inetpub\wwwroot\multihome\index.cfm: line 21
13Called from C:\Inetpub\wwwroot\multihome\application.cfc: line 228

Did you get anything out of that? Neither did I. The first thing I did was go review the scripted 'new Query()' documentation on the Adobe site. That didn't help. According to the documentation, everything appears to be fine. The next thing I did was look at other examples out there. Yep, still good. Next, I started comparing to other instances of code that I know to work. Wait a minute....Look at this:

SQL code with whitespace characters

See anything odd? Yes, I show whitespace characters in my editor (ColdFusion Builder). Pretty easy to do. Just go to your preferences and change it: go to Window | Preferences | General | Editors | Text Editors, and select Show whitespace characters. What you see in this picture are tabs, spaces, and End of Line markers. "OK, so what?" Well, here's where it gets strange. Let's take our original query, and put it all on a single line:

view plain print about
1var sql = "SELECT u.userID, u.username, u.password, u.dateCreated, u.lastUpdated FROM users u WHERE u.username = :username AND u.password = :password AND u.isActive = 1 ";
2var q = new Query(datasource = VARIABLES.instance.dsn,sql = sql);
3q.addParam(name = "username", value = ARGUMENTS.formScope.username, cfsqltype = "cf_sql_varchar");
4q.addParam(name = "password", value = ARGUMENTS.formScope.password, cfsqltype = "cf_sql_varchar");

If you run this, all is well:

CFDump 3

As you can see, everything works fine now. But, when I format my SQL for readability again, I again get the error. When I went back, and looked at examples that worked (in my editor) I discovered that lines following lines with params were directly preceded with one or more spaces. So, just to test, I added a single space right before those lines that followed lines referencing params:

view plain print about
1var sql = "SELECT     u.userID,
2                    u.username,
3                    u.password,
4                    u.dateCreated,
5                    u.lastUpdated
6            FROM    users u
7            WHERE    u.username = :username
8             AND    u.password = :password
9             AND    u.isActive = 1";
Code with adjust whitespace

This took care of it. My error went away, and my query executed properly, and my query was still formatted for readability. After another round of the great Tabs vs Spaces debate at work I had to change my editor's default preferences back, which is what caused/highlighted this issue. (I told you guys we needed to stick with 4 spaces ;) Maybe it's a bug in the SQL parser, or there's a method to the madness, but adding that single space before those lines is all that's required to get back on track.

ColdFusion 9 Hotfix 2 Released

Adobe has released the ColdFusion 9.0.1 Hotfix 2, available on the update page. This is a cumulative hotfix, containing fixes for security issues, items around ORM, resolution to questions of JSON serialization, integration bits for Exchange, and much more.

Install has some quirks. It's not just a simple 'upload the file' bit, so you'll want to pay careful attention to the instructions, and backup affected files in advance. This will get so much easier with the next version of ColdFusion, but for now it's worthwhile to jump through the hoops. It is a 'cumulative' hotfix, and word from those in the know say that it is safe to skip over the CHF 1 install, if you haven't done it already, as all it's changes are within this hotfix as advertised.

ColdFusion Hotfix 2 Released

Yesterday, Adobe released their second cumulative hotfix for their popular ColdFusion web application server platform.

Cumulative Hotfix 2 is the result of thousands of hours of review, development, and testing by the ColdFusion engineering team. These guys (and gals) review every bug submission, ask questions and gather feedback from many people within the CF development community, and work hard to address the issues in a timely manner.

Hotfix 2 covers major and minor issues dealing with JSON serialization, ORM relations, Solr indexing, cross-site scripting security, MS Exchange integration, and much much more. See the change list for more information.

A big thanks to the Adobe CF Engineering team for their ongoing, and ever increasing support, or the web's first application server.

More Entries