Essentially you should take heart in the fact that whilst the selected schedule *might* have been better by your inclusion (yes we know you are awesome and have written a gazillion books and are a respected MVP/ MCM and Certified Guru), but at least someone else will get the opportunity to shine (and hopefully be awesome too). Ultimately (in my opinion) the Summit is about a team effort so we should be nothing but pleased for them and take our failures in good grace.

The Summit 2013 marks my third year at the biggest SQL Server Conference in the World as a speaker in any capacity, my second appearance giving a regular session and is the third regular session in total that I will have presented there (I gave two last year). It will also be my fourth as an attendee. I have been very grateful to have presented two lightning talks in total at the last two Conferences and I am always very humbled to get chosen in any capacity and grateful to the people who make this happen. I would also like to give a very quick shout out to my good friend Niko Neugebauer who I was hoping to co-present a Personal Development session with. Unfortunately this session was not chosen which is a big loss for the Summit in my humble opinion since (for me at least) Niko oozes #awesomesauce and I think our session would have ROCKED! Still perhaps another time and I keep my fingers eternally crossed that Niko (blog|twitter) is successful at next years Summit if not sooner. If anyone deserves payback for all his HARD community efforts and major recognition for his abilities, he is your man!

I shall be presenting on one of my favorite subjects -SQL Server Concurrency. Those of you who follow me a little closer than is healthy will already be aware, last year I presented a session titled “READPAST & Furious: Transactions, Locking, and Isolation (DBA-309)” which focussed on some of SQL Server’s crazy behaviors and concurrency basics. My session this year is called “Lock, Block and Two Smoking Barrels – SQL Server Concurrency” and will follow along similar lines looking at even more SQL Server crazy behaviors but focus in a little bit more depth on transactional processing theory and contrast SQL Server’s Pessimistic and Optimistic isolation levels and how they may help/ hinder/ make no difference to the concurrency of your environment. The “Two Smoking Barrels” referred to in the title is of course a direct reference to SNAPSHOT Isolation Level and READ COMMITTED Snapshot Isolation -the optimistic implementation of READ COMMITTED Isolation Level. Yes we are sure to focus more on these two optimistic bad boys. I hope you can manage to come to my session, I think we will have a blast!

If you are interested, the following is the session abstract:-

SQL Server is a high performance relational engine and provides a highly scalable database platform but due to its complexity and bad programming practices can be prone to serious concurrency problems, unexpected behaviors, lost updates and much more!
In SQL Server 2005, two optimistic concurrency mechanisms were introduced and touted as the solution to all our problems. Now in SQL Server 2012 (and beyond) even more have followed, but many challenges and problems still remain.

Let’s take a long look into the world of SQL Server concurrency and investigate Pessimistic and Optimistic isolation understanding how they work, when you should use them, and more importantly when they can go very wrong.

Don’t be staring down the wrong end of SQL Server’s two Smoking Barrels and join me for this revealing and thought provoking presentation.

This time around T-SQL Tuesday is hosted by Bob Pusateri (blog|twitter) and his chosen topic is “how did you come to love presenting?”. So here goes…

In 2010 I had traveled to my very first conference as a presenter. So there I was, stood on the stage in front of a motley collection of SQL professionals, realizing the enormity of what was about to happen.

For many years I had been one of the guys in the crowd of a sea of attendees who had come to listen to awesome presentations from awesome professionals. From time to time I would wonder what it must be like talking in front of all those people -and more importantly whether I could be capable of such a feat. Did I have anything to say? Yes absolutely, but these guys were (of course) in a different league from me, and a league in which I could never hope to emulate. Ever. Then a funny thing happened one day… I decided to submit a presentation to SQLBits 7 really out of curiosity more than anything.

Oh Flip!

Months later, oh joy of joys… I got selected!

Finally when the euphoria faded, the reality set in. OH MY GOD… I’VE BEEN SELECTED!!! WHAT AM I GOING TO DO!!! This presentation was going to be my first ever presentation to my first ever audience anywhere ever and I knew it wasn’t going to be a walk in the park. There was also an extra element of excitement/ fear in that SQLBits always recorded their presentations, hosted them publicly for free on the internet, and (if that wasn’t enough) used picture in picture recordings of the presenter in sync with the slidedeck.

Yes people, even back then SQLBits was a very slick conference, and if you are going to fail at presenting, it is perhaps not the place to start .

And let’s not forget the leap of faith that the SQLBits committee had shown by selecting me for this event. Whilst I was not entirely unknown from a social media perspective (I had been a active blogger and twitter user for several years by then) the fact remains that as a presenter I was (shall we say) a little inexperienced!

So moving back to stage where I was delivering my first ever session…

I remember looking at the audience quite early on around 5 minutes into the presentation and I clearly remember thinking I AM ACTUALLY DOING THIS!!! After composing myself, I continued the session and think it went quite well. Not only was it one of the most frightening, exhilarating, scary and rewarding experiences that I have ever had, the months and months of preparation leading up to the event improved my technical skills exponentially from where they had been before.

Now with every presentation I deliver, I always learn hugely from each experience.

After SQLBits 7 I decided that I wanted more and have gone on to present at every SQLBits since (to date). I have also delivered sessions to many major SQL conferences and community events around the world, seen places I would have never seen, have met people I would have never met and embraced the SQL Community in ways that I had never realized possible.

So to the SQLBits Committee who made it all this possible and started my enormously rewarding journey, I would like to say a special thank you!

SQLBits I think I ♥ you!

One of the biggest benefits to the latest release of SQL Server (and reasons to upgrade) is that Windows Server Core is now a supported option. For those of you who are not yet familiar with Server Core, it could be best described as a reduced GUI version of Microsoft Windows. In other words, if you like the command line then you are in for a very big treat! Regular visitors to these pages will most likely be very aware of my passion for Windows Server Core and more to the point running SQL Server on it.

I digress slightly, but the point is that because SQL Server can now run on within a reduced graphical interface environment, one large beneficial side effect to this is the portability of the database engine. More specifically we can “hack” SQL Server to run on more and more types of hardware and software. The following is a closely guarded secret by Microsoft, but with with a few simple hacks, re-compilations and know how, we can spoof SQL Server to run on several other Operating Systems.

Let’s get to this!!!

Now is probably a good time to stress that whilst I will reveal some of the magic involved in fooling SQL to run elsewhere other than Windows, I am currently bound into a Non-Disclosure Agreement with Microsoft and will therefore be careful not to break it. Therefore any information that I have compiled within these pages are public domain and can be found from various sources elsewhere. It is also important to stress that as of this time of writing, only the SQL Server features supported by Windows Server Core can be “ported” across to other environments -so if you are wanting to use Master Data Services elsewhere other than “full-fat” Windows, then unfortunately you are out of luck.
For a list of all the SQL Server features currently supported on Server Core you should visit “Install SQL Server 2012 on Server Core“.

Several months ago I came into possession of a lovely new Kindle Fire HD and as you may know, this tablet runs a customized version of Google’s Android Operating System. Having had some recent (but limited) success with running SQL Server on some other Android based Hardware I decided to try my luck on this nice bit of kit, and the following is how I did it.

Install Server Core

Probably one of the most time consuming parts of this whole exercise is the installation of the Server Core Operating System. This can be especially bewildering if you have never done so before, but a very good source of information is the MSDN article ”Install and Deploy Windows Server 2012“. One reason I chose Windows 2012 as opposed to Windows 2008R2 was simply to eliminate any possible backwards compatibility issues with Android when we come to port SQL Server. For instance there is a well known bug in Android 1.6 Donut where a port of SQL Server from Windows 2008 will cause erratic buffer pool memory corruption.

Install SQL Server onto Server Core

Installation of SQL Server onto Server Core is relatively straight forward (see the previous link provided) as long as you are comfortable with performing a command line installation -either through using a configuration file or a full  set of command line options. Unfortunately (and perhaps understandibly) GUI based installation will not work, however I did identify a “bug” which allows a workaround which I detailed in the following post “Using the SQL Server Installation Wizard on Server Core!“.

Dump the SQL Server Process to disk

There are lots of ways to dump a process working set (the memory a process is consuming) to disk. On Server Core, it is no more difficult to do so since the Task Manager is (partially) supported and works as you would expect. You simply need to locate the SQL Server Service process, right click it and select the Create Dump File option.

Once this is successful you will see a message telling you where the dump file has been created. One very important consideration for you is to make sure that you have set a fairly low maximum memory cap for SQL Server’s buffer pool otherwise it the dump file will be much larger (and take longer to output) in addition it will make your ported SQL Server application to run faster.

Rename the dump file to .src

A fairly simple but necessary step to perform is the renaming of the dump file that we have just created. The reason for doing so is so that we can recompile the assembly in the next step and target our assembly for Operating System of choice. Simply right click the dump file and rename its extension to .src.

Recompile the source file using the .NET Native Image Generator

The Native Image Generator tool (ngen.exe) is an often overlooked and ignored utility for the .NET Framework but is very useful for improving the performance of our .NET applications and avoiding JIT compilation. There are several undocumented options for ngen and we shall use them to port our application. Since SQL Server is not a full .NET application and only uses parts of the .NET Framework (for optimization reasons) we can use the undocumented “/partial” switch which will recompile all necessary IL code sections of the dump file and will translate all other machine code sections into one complete assembly. Since we are also moving to a different Operating System we must also use the undocumented “/architecture:ux64″ switch to target a 64 bit Android environment.

Once we have created our native Android image of SQL Server there are only a few final steps remaining!

Install FSExplorer on your Kindle

In order to copy our SQL Server Assembly across to our Kindle Fire HD, navigate to the root and create a new folder we need a decent file explorer application. Probably the best I have come across on the Android Operating System is a nice little free app called FSExplorer. Open the Kindle App Store, search for and install FSExplorer.

Move the SQL Server image to your Kindle

Using FSExplorer, navigate to the file system root and create a folder called SQLSrvr where you will stored the Android SQL Server assembly. Copy the recompiled SQL Server image to your Kindle under this folder.

Set SQL Server to auto-load

In order to auto-load SQL Server as a daemon process you should first create an auto loader file under the root calling it startp (if one does not already exist). Make sure the file has the execute permission assigned and within it enter one simple line start /SQLSrvr/sqlsrvr.d.

Don’t forget to edit the startp file and copy the SQL assembly!

After you have made this last change, reboot your Kindle Fire HD and SQL Server should launch successfully.

Enjoy!

As it turned out I realized pretty quickly that I didn’t need to bother since the VM also comes with a copy of SQL Server 2012 Evaluation Edition media pre-loaded at the root of the C:\ drive. All I had to do therefore was to grab an existing Developer Edition license key and run the Evaluation Edition media setup and step through the Edition Upgrade to “convert” my existing instance.

However, I also knew that there were other utilities and tools (and possibly other installation media) that I would want to bring into my Azure VM and it all felt slightly restrictive. After checking out the Windows Azure Management Dashboard I could not find any options whatsoever to upload files to my VM.

Even after a little digging around in the usual search engines, the most useful suggestion I managed to locate was to download and install an FTP server onto the VM itself and play around a little with port settings! Whilst feasible, that level of tinkering was surely ridiculous. After a few moments of frustration I had an idea. Since I was connecting to my Windows Azure VM over RDP (and knowing that my RDP client is able to provide remote access to certain local resources such as the clipboard I decided to try taking the easy route and simply copy and paste an executable after enabling this option. To do so, you simply have to:-

• Launch your RDP client (mstsc.exe in Windows) and expand the details button to reveal the remote computer access to local resources checkboxes.
• Ensure that the Clipboard option is checked and connect and log into your Azure VM.
• Copy the files, folders or ISO from on your local host.
• Paste into your Azure VM.

Remote Desktop can provide access to local resources!

Whilst I have seen this fail in certain restrictive network configurations, to my delight SUCCESS. I know this operation is hardly rocket science, but sometimes the most simplest things are the hardest to figure out. Now my Azure VMs suddenly seem to be far more useful and I can’t wait to play further!

After a very busy start to November in which I delivered a presentation to my 2nd US SQLSaturday (and my 5th SQLSaturday of the year) in Portland, I then flew back up to Seattle to present two regular sessions and a lightning talk. Whilst 101 things could have gone horribly wrong with the whole affair, I think that I not only managed to pull it off, but also enjoyed the experience of doing so. Along the way I have not only met some fabulous new people but have also got reacquainted with many old friends.

Normally, all of this alone would have been a fantastic end to the year but there was lots more to come on my return back to the UK.

Within a week or so of arriving back in Cambridge from Seattle I received the results of the MCM Knowledge exam that I’d taken prior to leaving for the States. I had hoped that my results were going to be received prior to setting off for the PASS Summit so that I could take advantage of being there and scheduling in my Lab exam, but sadly this wasn’t to be. Still, I was very pleased that when they did arrive, to find that (joy of joys I’d managed to pass the Knowledge exam first time).

Whilst I didn’t think the questions were particularly hard, they were a little bit of a surprise (and truth be told I could have been a bit more prepared). In my opinion, the questions seemed to be not quite as polished as I had expected and a little unrefined or ambiguous (just my opinion -not necessarily fact).

Pretty in Pink

A week later I then received an email from PASS HQ stating that I was the recipient of their Outstanding Volunteer Award which is sparing given to just one (or a very small handful of individuals) every month GLOBALLY! This really was a very big honor for me and it was very gratefully received. Now would probably be a good time to state that the award is really something that I feel has been made possible by not just my efforts, but the efforts and sacrifices of those close to me (i.e. family and friends). To be allowed to spend a week in Amsterdam simply to help represent PASS at TechEd Europe this year is no small ask (and especially hard on the kids) but when you realize that this was simply just one of a multitude of trips away this year that contributed to being awarded (which have been very hard to professionally and financially justify), then you will probably agree that the award is more for them than for me. To top it off, my wife had been fundamental to the success of SQLSaturday #162 Cambridge in the UK this year having expertly project managed it to success amongst filling a thousand other roles. So a HUGE big-up to her.

Finally, the last Christmas cracker to arrive on my doorstep was the confirmation that I am now officially taking over the role as PASS UK Regional Mentor from Community good guy, SQLBits organiser and SQL Server MVP Mr. Chris Testa-O’Neill (blog|twitter). Yes, his are very big boots to fill but I shall try my hardest and will always do so with the interests of the Community at the forefront of my thinking. Chris will of course continue to support the community efforts (as he has always done) so I would just like to say a huge thank you to him for his contribution so far. His efforts also highlight the fact that in order to contribute towards anything, all we need is a desire to do so -and this desire is something which I believe I have in spades.

Obviously the decision for the Regional Mentor position has not come out of the blue and has been thought about and discussed for a very long time; and with a number of other worthy candidates here in the UK I can only be humbled by PASS’ choice of myself to work alongside (the other UK RM) Jonathan Allen (blog|twitter). Hopefully we can deliver exactly what is expected and much MORE!

So thanks to everyone who has contributed to all these things and more (a special mention should go to Karla and Niko). Hope you all have a fantastic Christmas!

What I am about to discuss is probably not the biggest secret to many of the “more informed” of you, and over the years many fantastic speakers (such as Kalen Delaney blog|twitter) have revealed the truth about SQL Server’s transactional behavior.

I first presented this subject (and this behavior) at SQLBits 9 (Liverpool) in 2011, and have subsequently presented the subject a few other times elsewhere (most recently at SQLSaturday #162 Cambridge), and each time never usually fails to spark the alarm of at least one person in the audience.

When I was a relatively new DBA (and I do have to go back a considerable time) I struggled for a very long time to understand exactly what was the point of SQL Server’s SET XACT_ABORT ON statement…

…I mean, as I understood it, if transactions failed (and therefore aborted) everything was rolled back anyway wasn’t it? So what would be the point of setting XACT_ABORT ON then? This statement just didn’t make any sense to me.

Then one day an incident happened at a company I was working for at the time, in which a massive amount of data had gone missing whilst it was being processed by an in-house application. The application would collect the data from a third party vendor, shred it and do funky things to it, before finally placing the data into a SQL database. Upon committing this data, the application would send an acknowledgement message to the third party vendor to provide a confirmation the data had been successfully received and could be removed from their queue.

When the missing data was detected by the application developers, the predictable finger pointing ensued. They blamed us for OUR crummy database and obviously misconfiguration server, and we blamed them for THEIR atrociously written application that was somehow (incorrectly) deleting data.

A typical transactional scenario

We have an orders table in which we wish to remove a record that needs to be fulfilled and insert the (fulfilled) record into the fulfilled table. If the insert fails, it is important that the delete is rolled back and if the delete fails it is important that the insert is rolled back.

The code for this is as follows:

DECLARE @orderid INT
SELECT @orderid = MAX(id) FROM orders
BEGIN TRAN
   DELETE FROM orders WHERE id = @orderid
   INSERT INTO orders_fulfilled VALUES(@orderid, 4, @@SPID)
COMMIT

So in the code snippet above you can see that we are deleting the top orderid from the orders table and inserting it into the orders_fulfilled table. If we execute this code, it does exactly what it says on the tin – deletes the record from orders table and adds it into orders_fulfilled table all within a transactional context. Pretty basic huh?

Now if we rerun the code above, simply changing line 5 as below:

INSERT INTO orders_fulfilled VALUES(@orderid, 5, @@SPID)

The following error occurs…

(1 row(s) affected)
Msg 547, Level 16, State 0, Line 5
The INSERT statement conflicted with the CHECK constraint “status”. The conflict occurred in database “READPAST & Furious”, table “dbo.orders_fulfilled”, column ‘status’.
The statement has been terminated.

So the error message itself is remarkably explanatory. It has told us that our insert statement failed due to the presence of a CHECK constraint on the orders_fulfilled table. Upon checking for open transactions, we note that there are none. Great, this must mean of course that our transaction will have automatically rolled back and undone our delete -right? WRONG!

Upon further inspection we note that not only did the insert not complete and that the transaction is no longer open, we observe that the delete did actually occur! The result if you were not expecting it is lost data.

Going back to our in-house application, the developers had introduced some code which was shredding vast volumes of data and performing a similar transactional delete and insert, but had not accounted for the fact that the constraint might be broken by the vendor data OR what would happen if it was. Because the application was acknowledging the receipt and commit of this data to the third party, not only was the data failing to insert into the database, but was also being removed from the vendor’s transmission queue.

Why is this happening?

This is not a bug in SQL Server, far from it. In fact this behavior is by design and occurs only when certain errors are raised within transactions. As you have seen, constraint errors are one such situation and the others that you may run into are (as far as I know still not documented in one single place).

I believe the thinking behind this behavior is to avoid unnecessarily undoing work performed by all the successful statements, which could come in useful in certain expensive transactional loading operations. Besides…if you didn’t want this to happen, then you would have implemented an alternative right?

What is the solution?

Well I’ve already mentioned the XACT_ABORT ON statement, and now it suddenly makes sense doesn’t it? Since we have shown that transactions are not truly “all or nothing” operations by default, we need a way to direct SQL Server to make them atomic. XACT_ABORT ON informs SQL Server that it absolutely *must* rollback any transactions if there are ANY errors during their lifetime or commit everything.

There is a second and more common way of making transactions behave as expected and this is by using error handling. SQL Server 2005 introduced the BEGIN TRY and BEGIN CATCH syntax which hugely simplified our transaction management. Within our catch-block we can simply perform any clean up actions that we want and ROLLBACK our transaction should we so desire.

So what happens if we use both methods together?

There is a particular problem to be aware of when using error handling in conjunction with XACT_ABORT ON and this is really due to the implications of setting this behavior on. By doing so, as we have already discussed, you have told SQL Server to COMMIT everything or roll everything back. When using error handling you are enabling functionality to take custom actions within your transactions. One such action could be a rollback to a savepoint, or perhaps to insert logging information to another table.

Let’s see what happens with the following code snippet:

SET XACT_ABORT ON
DECLARE @orderid INT
SELECT @orderid = MAX(id) FROM orders
BEGIN TRY
   BEGIN TRAN
      SAVE TRAN savetohere
      DELETE FROM orders WHERE id = @orderid
      INSERT INTO orders_fulfilled VALUES(@orderid,5,@@SPID)
   COMMIT
END TRY
BEGIN CATCH
   ROLLBACK TRAN savetohere
END CATCH

The following error occurs…

Msg 3931, Level 16, State 1, Line 15
The current transaction cannot be committed and cannot be rolled back to a savepoint. Roll back the entire transaction.

The error we are experiencing here is known as an “uncommitable transaction” and occurs because XACT_ABORT is explicit in requiring that the whole transaction is rolled back on error. Performing ANY operation that causes log writes (such as an insert, delete, update or even rollbacks to savepoints) will result in error. It is not obvious from the error above, but rollback is automatic and no transactions are left open when this condition is hit.

So should we ever use XACT_ABORT ON then?

There are a couple of situations that using XACT_ABORT ON is the right or necessary thing to do (with or without additional error handling). One such situation is when you have unskilled (or lesser skilled) labour executing and promoting SQL scripts into your environments. You (or your developers) want to ensure that the scripts either work OR they do not. If you use distributed transactions then you will be required to use XACT_ABORT ON.

In the scenario that you do use both techniques, it is important that within your catch-block you detect any uncommitable transactions before attempting to perform any logging operations. This is performed by detecting the transaction state and the XACT_STATE() function enables us to programatically do this.

For instance:

IF XACT_STATE() <> 0 ROLLBACK TRAN --if there is an open transaction, roll it back!
--if tran state is -1, then tran uncommittable, if 0 then no tran open, else if 1 tran is committable

Final Thoughts

Understanding SQL Server transaction management is far more difficult than you may realise and it is important that you know the implications of how your developers implement them. In the scenario presented here, we have demonstrated that if they/ you get things wrong then your database may be losing data without you even realizing it. The use of good T-SQL error handling and transaction state detection can be the difference and very simple to implement in the right hands.

Whilst it is common in organizations for Developers and DBA’s to point fingers at each other when problems are experienced in your environment, it is important to understand that this usually happens due to poor communications between the two parties and most importantly because you both care. It is YOUR job as a DBA not only to protect your data but also ensure that you educate Developers where possible. Likewise it is the Developer’s job to ensure performant and accurate application code and educate DBAs. Understanding each other’s worlds better and communicating more efficiently can avoid some of the largest (and sometimes more costly) problems with very little effort.

SQL Server Lock Escalation is one of these commonly misunderstood topics, and most recently after doing a little investigation into a few subjects related to escalation I was shocked and surprised in equal measure to find that things were not working quite as I expected.

As most of you will know, the reason SQL Server has an escalation mechanism is to provide a balance between concurrency and efficiency. On 64-bit SQL Server, since each lock consumes 128 bytes of memory and lock owner block (process holding the lock) consumes 64 bytes of memory, locking can start to consume an awful lot of memory and have an impact on system performance if escalation did not exist.

By default, there are really two reasons why escalation will occur:

1. A result of too many locks taken out on a resource by a statement (the ball-park figure is just over 5000).
2. The second is due to the amount of lock memory consumption (roughly just over 40% of the current SQL Server working set).

So let us assume that we have a transaction where we are making updates to many rows in a table.

For example:

BEGIN TRAN
UPDATE database1.dbo.table1
SET C2=1 WHERE C1 < 1000;

Now I don’t think it will be a huge surprise to anyone when we see that we have taken 1000 Exclusive (X) locks and 3 Intent-Exclusive (IX) locks.

So now what happens if you run the following statement inside the same transaction:

UPDATE database1.dbo.table1
SET C2=1 WHERE C1 >= 1000 AND C1 < 2000;
UPDATE database1.dbo.table1
SET C2=1 WHERE C1 >= 2000 AND C1 < 3000;
UPDATE database1.dbo.table1
SET C2=1 WHERE C1 >= 3000 AND C1 < 4000;
UPDATE database1.dbo.table1
SET C2=1 WHERE C1 >= 4000 AND C1 < 5000;
UPDATE database1.dbo.table1
SET C2=1 WHERE C1 >= 5000 AND C1 < 6000;

Just looking at the whole transaction statement you would assume that since around 6000 locks are being taken on the same resource we would get escalation occurring. Let’s take a look:

As you can seem, escalation hasn’t happened. This can however be explained by the statement I made earlier, that escalation will occur if the threshold is exceeded for a single statement on a single resource. I have performed tests and found that there does not appear to be any upper limited on the number of locks you can take on a single resource as long as each statement does not break the threshold and the memory threshold is not broken.

So now let us try to get escalation occurring. We shall first execute the following in the same transaction:

UPDATE database1.dbo.table1
SET C2=1 WHERE C1 < 6000; 

Clearly you can see that we have “broken” the number of locks limit. However the results are identical as before. Since the locks have already been taken by this process within the context of the same transaction, the threshold is not broken and therefore escalation does not occur. This is an important point to raise in that the threshold test is evaluated once the decision to acquire locks has been made. Finally I shall run the following statement (again within the same transaction):

UPDATE database1.dbo.table1
SET C2=1 WHERE C1 >= 6000 AND C1 < 12000;

We have now issued a statement to update 6000 (new) rows, this statement is still being run within the same transaction on the same resource and as such we would expect it to trigger an escalation.

This time escalation of all the Exclusive key locks and Intent-Exclusive page locks have been successfully escalated to a single object level Exclusive lock as expected.

Conclusion

When I first saw this escalation behavior I was of the opinion that it was indeed a bug that I had uncovered. Over time and reflection I believe that it is not so much a bug as it is an oddity. The stated behavior of SQL Server to escalate due to the number of locks on the same resource by the same statement is being adhered to, although as I pointed out, if the locks have already been acquired, then these do not count towards the final threshold evaluation.

You could argue that perhaps SQL Server should have an optimization to cumulatively add the total number of locks owned by a SPID on a resource and compare this figure against the threshold limit, but I assume the Database Engine team must have ruled that out for concurrency optimization.

Can you use this oddity in a useful way? Perhaps…
Lets assume you need to perform a large data set change on a very large table or partition which is highly active. The data must be changed and performed whilst your environment is live, but you wish to minimize the impact to the table’s concurrency. (I should also add here as an aside that escalation is not guaranteed if incompatible locks exist when it is attempted, but do you really want to run the risk that it could happen in this proposed scenario?)

Your first thought may be to change the data in batches, but perhaps you require that the whole operation is transitionally consistent? All of the dataset is updated OR it is not.

In this situation you could break down the updates into logical statement batches, ensuring that the number of locks taken by each does not cross the threshold. These statements can then be run inside an explicit transaction and escalation will not occur (or be attempted) due to thresholds not being crossed. Therefore you will only lose concurrency to those rows themselves and not require or attempt exclusive partition or object locks. And whilst it is possible to turn off escalation completely due to the number of locks (TF1224) or on the specific table itself (as of SQL 2008), this is not always possible or desirable in certain environments and this method could prove to be a far more satisfactory way around the problem.

One Final Note

Hopefully this post has demonstrated that SQL Server mechanisms do not always work the way you expect (or have read in a book). With a little bit of testing and observation, it is usually possible to understand what is really going on. So don’t delay and dive into your test environment!

As a fairly regular speaker myself, I have experienced many events who have either made very little effort with the dinner, poorly handled its organization (meaning many speakers aren’t even informed that it is happening) or simply don’t bother arranging one at all. And whilst I don’t want to suggest that by organizing an event you absolutely MUST provide a speaker dinner in order for the event to be regarded as a success, I do personally feel that the speaker meal helps define your event and give it a feel or your stamp -and hopefully one along the lines that you were aiming for.

A great sense of humor to be had in Dublin!

A very good example of this can be found by contrasting a couple of the SQL Saturday events I have attended as a speaker this year -specifically SQLSaturday #115 Lisbon and SQLSaturday #105 Dublin. On the Friday evening in Dublin, they hired a barge that sailed the length and breadth of the Canal whilst serving up the meal. We all ate and drank merrily, chatting as the banks slowly passed by. It was not only a novel and unique idea, but was great fun and made us all feel quite special. This extravagant touch reflected the professional and expensive polish of the Dublin event, and whilst I do not normally enjoy too much fuss I confess I did enjoy it enormously.

In Lisbon, having experienced an incredible attentive touch from the organizers on arrival, and leading up to the Saturday, we effectively had not one but TWO speaker meals! The first was held in a rather incredible ornate building -which as I understand it was once a Palace owned by Portuguese Royalty. Amusingly it had so many dining rooms, myself and Rob Volk (twitter|blog) could not find Niko (twitter|blog) and Crew at first and wandered aimlessly between each room until Niko came hurtling out of one of them to call us in. The second meal was hosted during Saturday lunchtime in a very traditional Seafood restaurant where upon finding out I was presenting directly after, Allan Mitchell (twitter|blog) relentlessly tried to encourage me to drink lots of Red Wine to “settle my nerves” as he put it .

Lisbon was an incredibly beautiful city

Both meals were enjoyable and each had a very authentic ambiance. So which event meal was best (Dublin or Lisbon)? For me each one of them had its merits and each one set the scene for the SQLSaturday to come. Dublin demonstrated that you were speaking at a very professional well run event and provided a hint and feel of Ireland. Lisbon immersed you into the proud and welcoming Portuguese culture making you feel not only very honored to be there and in the company of friends, but to also gave the impression of being on holiday.

It was with these two events in mind that we would go onto organize and shape our own speaker meal. We wanted to provide a mixture of a relaxed down to earth friendly atmosphere and a very well run organized (and well attended) speaker meal. During the last four weeks leading up to SQLSaturday #162 Cambridge, myself and the event PM (Lorraine) decided that if we were going to provide an evening meal to 30-40 people, we had better make sure we knew beforehand if it will be any good. Initially (and ideally) we wanted to have a very English style meal and ambiance, but the first few restaurants we had tested had not really come up to the bar that we had set. La Strada, our third restaurant, and I am sure you can guess by the name was anything but traditional English gave us a fantastic meal one lunchtime and had been very attentive. They also had the perfect area that could be reserved for us all which was slightly tucked away from the other restaurant goers but not entirely. This meant that we had the best of both worlds in this respect. After chatting with the restaurant manager we knew that this was definitely the best choice for our dinner and booked it up.

Bond: Mr. Woody, do you expect me to talk?
Woody: No Mr. Bond, I *expect* you to eat your pie!

We were very pleased that out of all the speakers, nearly every single one confirmed (apart from one I believe). We did have a couple of speakers coming from Portugal (yep that guy Niko again …and the excellent Bruno Basto) that had indicated that they may be very late and might not be able to attend, but that aside we had an excellent turn out. We had also decided that every Gold and Silver sponsor (should they not already have speakers at the meal) would be given up to two invites for their representatives. We were really happy that every single sponsor took up the invite boosting the numbers further. Finally we were very honored and proud to be able to invite and include a couple of VIP guests (Marjorie, Christina, Magda & Co!). These were friends, family and partners of a few of our dinner guests and I believe this really helped make the atmosphere even more relaxed and friendly.

Myself and Lorraine had discussed at great length about producing a seating plan for everyone, but at the last minute decided to “go with the flow” and let people sit with the people they really wanted to. Again I think this generally worked really well and contributed to the atmosphere.

So what went wrong?
Personally I felt that the evening meal was a huge success. There were however, a couple of points of note. The first problem was that we delayed the order until everybody had arrived. Whilst we had given around 30 minutes gap between seating and serving the order, we still had several people arriving slightly later. After another 10 minutes we decided that we couldn’t wait any longer and gave the word for the Chef to get his groove on. Thankfully by this stage, we were only around three people down (and two of those were the expected delays). At the time I was really conscious that the delay was slightly longer than people would have liked but it did give them time to chat, relax and drink wine -and certainly nobody had revolted at this point.

Another area of difficulty was encountered when the food was being served up. Since no one was sat in allocated seats, somebody needed to direct the (pretty!) waitresses to the right people/ tables. Yes it was a dirty job, but someone had to do it, so I volunteered.

Despite my ineffective direction, they did a fantastic job of serving up, and learnt very quickly to ignore my help. Once the dishes started rolling, they kept coming at a pace and everyone seemed to be very happy with the food.

Bleary eyed and bushy tailed, Moi and PM Lorraine starting to think about tomorrow and an early start!

My two Portuguese buddies arrived just as desert was being served and I asked Bruno whether they wanted something to eat to which he replied “of course!”. So after I managed to get them both seated, the restaurant staff did an excellent job serving them with vigor so that they were not waiting too long between courses.

Finally by the end of the night, the time came to settle the bill. Whilst it was rather large (four figures) it was almost exactly what I had expected and what we had budgeted for. Just as I was settling up, Lorraine reminded me about a tip. I’d totally forgotten about that, but they had certainly earned it. She told me that she probably had £10 in her purse, completely forgetting the size of the bill! After instructing the waitress to add on the correct percentage (yes it was much larger than £10), we departed for a relatively early night in preparation for the UK’s FIRST EVER SQLSATURDAY the following day!

The next post in this series will be “Memoirs of SQLSaturday #162 Part III: The SQLSaturday”. The previous post “Memoirs of SQLSaturday #162 Part I: Pre-Cons” can be found by clicking on that url.

With SQL server 2012, Windows Server Core became a supported option (on versions 2008 R2 and above) and whilst not every single component of SQL Server is supported on Core, there is now a very strong argument to move to that flavour of the Windows Operating System should your scenario warrant it. In fact for the last 12 months or so (and in particular when I am presenting my “Enter the Dragon: SQL 2012 on Server Core” presentation, I always like to try and make a point to the audience that if they do not yet know what Server Core is, then they very much will do in 2 years time. Love it or loathe it, in my opinion, they *will* be using it by then. I predict a massive shift to what I like to call “Skinny” (or “Low Fat“) Windows as opposed to “Full Fat Windows“.

For those of you reading this article who are not entirely familiar with exactly *what* Windows Server Core is, I shall describe it simply as “a reduced graphical interface version of Windows“. Essentially what this means to you as an IT Professional or DBA is, that certain applications may or may not work on Core -especially if they use a Windows based GUI. This does not mean that all GUI based apps will not work, just the majority. Simple GUI driven applications such as Notepad.exe or Taskman.exe have full, or in the case of Taskman -partial support. There are a couple of technical reasons why many GUI driven applications many not work on Server Core, but this is outside the scope of this post and perhaps I shall write about that another time.

We move nicely onto the subject of SQL Server and its related technologies and functionalities that it provides -which are supported on core. By and large there is a number to choose from so running SQL Server on Core is a very real proposition. Essentially, if you are thinking of moving to Skinny Windows, then I suggest you take a look at the MSDN page “Install SQL Server 2012 from the Command Prompt” that lists the supported SQL Server features on Server Core.

By the time you are ready to install SQL Server on your Server Core Operating system, you will probably realize very early on that you have a little bit of a problem. Attempting to launch setup.exe will result in a fairly self explanatory error message.

Oh flip!

In fact I am sure you probably won’t have ever got to that error message because being the type of person I know you probably are, you will have read all the necessary literature and have noticed in the MSDN article “Install SQL Server 2012 on Server Core” it explicitly states “SQL Server 2012 does not support setup by using the installation wizard on the Server Core operating system. When installing on Server Core, SQL Server Setup supports full quiet mode by using the /Q parameter, or Quiet Simple mode by using the /QS parameter“.

There is a rather interesting parameter that you may see mentioned in the error message in the previous screen shot called UIMODE. Essentially if we pass in the switch and parameter of /UIMODE=EnableUIOnServerCore to setup.exe, this time we DO get the Installation Wizard. But (and this is a big BUT) none of the installation options will work.

Who needs working installation options anyway?

By now you are probably starting to curse, and wondering what is the point of that option then? Looking back at the error you will also see the hint about using the /QS (or /QUIETSIMPLE) switch which shows progress but does not allow input. So if you try to use /UIMODE=EnableUIOnServerCore to bypass the Server Core restriction and specify /QUIETSIMPLE to show progress you will surprisingly receive an error message stating that these options are incompatible with each other!

Ironically we can simply specify the QUIETSIMPLE parameter without UIMODE set and now see the installation progress but only for unattended installs using a configuration file.
e.g. D:\>setup.exe /IACCEPTSQLSERVERLICENSETERMS /CONFIGURATIONFILE=”ConfigurationFile.ini” /QUIETSIMPLE

We will return to the UIMODE shortly but it is worth highlighting that we are still no closer to using the SQL Server Installation Wizard on Server Core! We have proved that the GUI does seem to work (albeit in unattended mode) and you will be wondering why *if* the GUI appears to work without error on Server Core in unattended mode, doesn’t it work in Wizard based attended mode. The truth of the matter is that the SQL installer is blocking itself on the Server Core OS and I assume Microsoft are gently trying to nudge you away from this type of deployment and that way of thinking on Server Core.

Now what if I told you that you can?

I first stumbled upon this bug (for want of a better word) whilst heavily involved in a SQL Server 2012 writing project earlier in the year -it was loosely related to another problem and bug caused through the very same issue. For more details on the other issue you can read my Connect 713497. If you are performing a command line based installation of SQL Server (and Core will force you through the command line), there are essentially two ways in which you can configure the SQL Server installation. The first is through parameters passed via setup.exe (as we have seen above) and the second is through the use of a configuration file -the location of which must be passed through the /CONFIGURATIONFILE parameter from the command line.

Now consider the situation where we have one parameter passed through the command line and one configuration file setting for the same parameter, both with different inputs. The expected behaviour is for the command line parameter to override any configuration file setting, and this is exactly how it works and is very useful for what I refer to as “templated deployments“. Specifically, you template standard configuration settings in configuration files and override them where your deployment necessitates so that your builds only deviate depending upon their requirements.

some bugs can be useful

In the aforementioned connect item, you will notice that I discovered the bug in which specifying the short-cut equivalent of QUIETSIMPLE (QS) on the command line and providing an alternative configuration file setting using the long syntax, the short-cut does not override the long syntax in the configuration file. In contrast, the short-cut equivalent of QUIET (Q) DOES (correctly) override the long syntax in the configuration file.

One behaviour that I did not mention when I raised that connect item was that by using the /QS (command line) and QUIETSIMPLE (configuration file) setting is that we can confuse setup.exe to believe we have specified a unattended installation through the /QS switch. Because it fails to override the QUIETSIMPLE setting (we have set it to False in the configuration file) -we manage to fool the SQL installation, skip the block for Core and enable the Wizard deployment through that configuration file setting.

Cool eh?!

Now I have to confess that through my connect item you will note that it has been marked as fixed in the next Service Pack…. I might have single handily caused the demise of the most useful SQL Server 2012 bug (for Server Core deployments) by raising that connect. …Just call me badassss!

But have no fear. Whilst writing this post and performing a little testing I came across another bypass. Whilst I demonstrated earlier the relative uselessness of the UIMODE switch, and it is worth pointing out that according to Connect 706799 it seems the switch is not even a supported feature, we can actually use it to perform a similar piece of magic as before. If you remember earlier, when we used the switch and parameter of /UIMODE=EnableUIOnServerCore, we did not need to specify an unattended installation switch such as /QUIETSIMPLE or /QUIET. Unfortunately, when we performed this operation we essentially entered the rather useless setup screen. HOWEVER all we need to do is provide a switch to a configuration file ensuring that within it, we have set QUIET=”False” and QUIETSIMPLE=”False”. By doing so you will also skip the installation block and viola enter a fully functional SQL Server Installation Wizard on Server Core!

Enjoy.

p.s. Remember that Server Core does not support every single SQL Server feature, so it is important that you only select the supported features as documented by MSDN, otherwise the installation is likely to fail.

When I first set out on the path of recording what I view as very much a mixture of personal and technical journal in the pages of this blog (and hopefully at times a good resource for others), I always intended to write at least one post per month. I always viewed it as a cardinal sin to let things slip, but unfortunately I truly underestimated the enormity of the work involved in planning, implementing and executing a major IT event. Now it is almost over (there are still many small administrative issues to complete) I think it is fair to say that Team SQLSaturday #162 Cambridge are able to take a deep breath and be proud of what we did.

Buck Woody giving some sage career advice

In my last post “Change and make a difference“  I told my story of how I first had the idea of bringing SQLSaturday’s to the UK -which could so easily have been missed had I not made the effort to improve my social self and turn up to one single social evening at the PASS Summit in 2010. It demonstrates if anything ever could, that through the help and knowledge of others we can do beautiful things.

In this post I really want to start to review and celebrate the two days of this momentous event and we shall first begin by looking at the fantastic pre-cons on the Friday. I believed that if we were going to host a Friday training day as a pre cursor to the main Saturday community day, it was important to get some very talented people to present them.

We were first fortunate enough to land the services of one of the most well known and highly respected Microsoft employees -a certain Mr. Buck Woody (blog|twitter). After a lot of discussion with Buck about which pre-con to choose (yes he has several!) we decided to go with a rather fun idea and called it “Buck Woody’s Ragtime DBA Workshop”. I wanted this to be aimed not just at novice SQL Server DBAs, but wished to corner the market for Developers and Oracle DBAs who want to learn more SQL and perhaps even to help more experienced SQL DBAs. This really was a tall order to fill since we were trying to span a very wide audience skill range, but if anyone could manage it, Buck could.

My next choice for a pre-con was a very clever speaker from Denmark called Mark Rasmussen (blog|twitter) who has reverse engineered the SQL Server Storage format for a SQL Server MDF File Parser called OrcaMDF -it can read and query the data files without SQL Server! I have been privileged to get to know Mark quite well over the last 12 months or more having spent a lot of time with him at SQLBits and SQLPASS and knew from the very beginning of pre-con planning that he was someone I wanted to present. Whilst the topic is quite specialised and knew the market for the material was perhaps smaller than other more “in demand” mainstream subjects, it was really due to my desire to sit in his pre-con (having already missed at least one opportunity to do so) and interest in the subject matter, that convinced me. If I wanted to attend, there would be many others -and that proved to be the case.

My final choice for a pre-con session is someone who has rapidly gained a very high reputation for her Data Visualisation knowledge and DataViz tool expertise. I am of course talking about Jen Stirrup (blog|twitter) -a friend I have known for a good few years. Having sat in many presentations she has given before, I have always been impressed by her ability to quote both popular and less popular authors on a vast range of subjects loosely related to DataViz and am never disappointed by what I see. This selection really wasn’t a difficult choice and provided a unique alternative to the other two offerings.

The feel of SQLSaturday #162 was always aimed at being very much an inclusive community affair and I was very happy that all pre-con speakers told me from the start that being part of the event was the most important thing.

Marco was sad he missed the Pre-Cons

One the day itself, I had a few minor hiccups with the venue, which with the help of two of our team and like me, fellow pre-con registrants John Martin (blog|twitter) and Mark Pryce-Maher (blog|twitter) we resolved these problems very quickly before people started attending. I think without John and Mark we would have struggled and I was very grateful to them for getting so involved when they had also paid to attend the sessions. I was also disappointed and surprised that we had three people who did not turn up to the pre-cons having fully paid up. I did not receive any advance notification, and strangely enough have still not received any word of what happened -but it introduced a problem in that I felt someone needed to wait on the registration desk for several hours to make sure that they were not coming. That person obviously had to be me (missing a large part of Mark’s session). I also needed to oversee a few minor administrative issues during the afternoon and I had to miss the last part of his pre-con. Whilst that was personally disappointing, it was probably no big surprise.

On the whole, the day went really well and everybody that spoke to me were very pleased with the Friday pre-cons and personally thanked us for putting them on.

In the next post of this series I will talk about the successful SQLSaturday #162 speaker dinner…