CFQueryParam problems after you change a query

On CF-Talk today, an interesting comment about CFQueryParam was made by Adam Howitt. He mentioned that sometimes you will run into database errors because of using cfqueryparam. He wasn't saying not to use it, I want to get that out up front. However, when you use cfqueryparam, ColdFusion creates a prepared statement (precompiled) to send back to the database. This makes things faster. However, sometimes CF will keep these prepared SQL statements in memory, even after you make a change to a query. So the user runs a page with a changed query, and ColdFusion tries to run the prepared statement from cache, error occurs. Adam has a work around, temporarily change your query to force the CF server to recreate your prepared statement. For example, add 'and 1=1' at the end, which doesn't change the result set.

I just wanted to post this here for future reference, because I always remember things better when I write it down. If anybody has any problems with the above scenarios, please leave a comment to enlighten me. :)

Comments
That is interesting. I have never run into that (and I use cfquery extensively). I would be interested in finding out under what scenario this occurs so that I could look out for it).
# Posted By Steve Bryant | 4/14/06 1:06 PM
Hi, The issue is fairly intermittent and depends on whether you have "maintain client connections" on which leads to the issue. If you change your underlying database structure (add a column, delete one) and you are using CF's JDBC drivers you sometimes see the cryptic error. For reference, it isn't always 19 >= 19. I am not certain but 19 represents the SQL Server prepared statement number. Turn on Query Profiler to monitor your database next time you get the error and see if the prepared statement number matches.

Adam
# Posted By Adam Howitt | 4/14/06 9:42 PM
I run into it quite often, and it is a pain. Create a view, alter a table in the view (add new column at the end). Reload the view. Refresh the page containing a query using the view. The data doesnt shift over and you have to come up with a way to reload the query. I often use -- #rand()# after the select to force the query to recompile. It does nothing to the query, because SQL reads it as a comment, but CF sees it as a unique query and forces a recompile. The downside, if you leave it in, it negates any performance gains from using query params.
# Posted By Michael | 4/14/06 9:49 PM
Thanks for verification of this Adam and Michael. I'm wondering if this is possibly a problem with SQL Server, and not ColdFusion. It seems like it's probably CF, but I'm just curious if this problem doesn't appear if one were to use Oracle, MySQL, DB2, etc.
# Posted By yacoubean | 4/14/06 10:47 PM
I've found this issue ONLY occurs when using select * with a query param.

Coldusion caches the order of the columns with a numeric index when selected with * and will not realize that the cache needs to be updated when you change the columns in your database or view. This is a particularly big problem when you add a column not at the end of the table or view, but in between other columns (as the types will then fail).

I have never seen this happen in any other case and I cfqueryparam every single variable I pass into my databases with no exceptions.
# Posted By Mike Kelp | 4/15/06 1:00 AM
Mike,

That's an interesting observation and it certainly fits with my experience. I have run into problems with queries using "SELECT *" when changing the columns in my table.

That is one of the reasons I avoid "SELECT *" and encourage developers with whom I work to do the same. (Ben Nadel details the other in the related discussion on CF-Talk).

This explanation would also be consistent with the problem reported on CF-Talk. Perhaps more experimentation is needed to verify?

For reference:
http://houseoffusion.com/cf_lists/messages.cfm/for...
# Posted By Steve Bryant | 4/15/06 6:08 AM
Amen to never using select *

I'm willing to claim that I've sufficiently verified it over about 3 years. I haven't used a select * in my final code in about 1.5 years now or more.

As an experiment try this:
Create a table with the following columns:
testID (int)
firstName (varchar),
lastName (varchar),
age (tinyint)

Enter some test rows and CFQuery this with a select * and at least one cfqueryparam in the where clause. Works great.

Now, add a column in between any of the others...This will almost always cause an SQL error that reports a type mismatch in the select statement, due to the order of the columns not matching the order of types that CF expects from the pooled statement.

Type the column names in and it all goes away no matter what you do to the table.

Anyway, I hope this is helping someone out. I need to get out on the lists more than just my local CFUG, but I haven't mastered the Sean Corfield "dual presence" technique that allows me to learn at lightning speed with a seperate thread dedicated to sharing haha.

Take Care,
Mike.
# Posted By Mike Kelp | 4/15/06 10:06 PM
I have encountered this problem all too often. I have even encountered a variation of the same problem where query results are returned in the wrong table column. For example, column A would return 'apple" and column B would return 'orange'. Once in awhile a query with cfqueryparam I will get 'orange' for A and 'apple' for B. the only way I have found to fix it is to remove the cfqueryparam from the query. BTW. I'm using CFMX 6.1.

My suggestion is to use data validation before the query to prevent malicious code from being entered into your query.
# Posted By David Levin | 8/15/06 6:12 PM
BlogCFC was created by Raymond Camden. This blog is running version 5.9. Contact Blog Owner