Lessons Learned: Upgrading Sitecore from 6.1 to 6.5

Posted on March 11, 2012 by Max

Just deployed the latest and greatest in Sitecore for my company. The guy on the left knows what I am talking about.

With the multiple languages and the complexity level of my company’s site, this was not a simple process of just running a couple of SQL update scripts and modifying some lines in the web.config. This upgrade took at couple of weeks and there were a lot of lessons learned along the way. Hopefully, my lessons will save you some time if you are going the same route.

By the way, shout out to @alexshyba once again, for pointing me to the right places and saving me hours upon hours of time.

Break Tag Embedding

Even though Sitecore documented this as a resolved bug, I still experienced this issue. If there are new lines in a rich text or multi-line field, HTML break tags (<br>) get inserted. While it’s usually hard to notice this, we keep XML schema in multi-line text fields, so any extraneous characters and the schema is broken.

The good news is, I caught this earlier and documented a fix for it here: http://blog.image0.com/sitecore/sitecore-multi-line-removing-br-tags/

Sitecore Query Performance

If v6.1 was forgiving with overly broad queries, v6.5 won’t be. It’s not out of spite or to teach you a lesson. I believe there was some security code added to the internal Sitecore query mechanism. There was a huge performance difference for me between the two versions – we are talking seconds, not milliseconds.

If you are on the same boat, you have 3 choices:

Optimize your queries
Consider using Fast Query instead of the traditional Sitecore Query mechanism (read my findings on it below before you do)
If 1 and/or 2 do not work, I’d look into using Lucene search.

Fast Query

It doesn’t take long to sell Fast Query over the traditional Sitecore Query. When I was writing a product recommender type app that generated results based on submissions, I was sold immediately by Fast Query. The caveat with it, of course, is that, in addition to returning unordered list results, it also doesn’t guarantee to return them in the context language. This means that if you are on a US site and you do a Fast Query, you may get some items from other languages (if they are defined).

I’m not sure if it was luck or otherwise, but on v6.1, I’ve never encountered that. Results were always from the same language. When upgraded to v6.5, every other query had mixed results.

There are a couple of different ways to achieve this with the Sitecore API in C#, but you basically want to check for 2 conditions -

Make sure the Sitecore Item’s Versions.Count property is greater than 0.
On the Sitecore Item’s Language.FirstOrDefault() language, do a Database.GetItem() passing in the Item ID and the context language.

If you check StackOverflow, there is actually a pretty good code block that does both steps here: http://stackoverflow.com/a/8232087

Multi-Server Publishing & EventQueue

The staging module ends up being history in v6.3 and up, which is great news if you have ever had to set it up for more than 1 server – a lot of tedious configuration and painful investigation of logs if something went wrong. In v6.3 and up, all you have to do is set EnableEventQueues property to “true” in your web.config, and it publishes your items and clears the cache correctly whether you have 1 or n balanced web servers.

Web.config Consolidation

My original web.config was huge in size and had a lot of country and environment specific values. For example, my UK domain on staging was different from my UK domain on production. Both had to be specified in the <sites> section of the web.config. In addition to it being a pain to manage it from a size perspective, there was also the risk of it being overwritten by someone who didn’t know any better.

For Sitecore configuration, your App_Config directory includes an /Includes directory. What a catchy name, right? In here, you can extract your Sitecore configuration from the web.config and then just dump it here. Just make sure your file has a .config extension for it to get picked up. The IntoTheCore blog has a really good post on how to do this here: http://intothecore.cassidy.dk/2009/05/working-with-webconfig-include-files-in.html .

There is a neat way of seeing exactly what gets cooked up at runtime, as far as your Sitecore configuration goes. Try opening up http://[yoursite]/sitecore/admin/showconfig.aspx to get an XML dump of your Sitecore configuration.

Please note that a) this only includes your Sitecore configuration, as opposed to <system.web> or other non-Sitecore related configuration data, and b) you can’t include non-Sitecore configuration in the /Includes folder. You can extract modules from the web.config if they have an configSource or a similar property – which is what allows your DB strings to live in ConnectionStrings.config as opposed to the web.config file.

Testing/Deployment Strategy

So, you recompiled your code with the new version of Sitecore.Kernel.dll and your site renders. Great. I wouldn’t flip the switch so fast. Keep in mind that for us, the majority of the duration for this upgrade went to testing and fixing issues.

What I recommend is having a QA team or at least a second set of eyes perform the regression tests side-by-side – run a box with v6.5 and have them compare it to your production environment (which is running v6.1 or something older than v6.4).

I had my load-balanced web servers, so to simplify deployment, I just created new blank databases for v6.5, published the entire site to them, and changed my connection string. That way, the v6.1 DBs and configuration was still intact on production.

When the testing was completed, all I had to do was clone the webroot on the CD to all its load-balanced brethren and that was it – zero downtime. Because…

A little planning saves a lot of fan cleaning.

Additional Lessons

- In a perfect world, we’d be able to freeze the environment – both code and content – and be done with this in 1-2 weeks tops. But, because we don’t live in a perfect world, there were changes that were constantly happening. For managing the code, we created 2 branches – v6.1 code branch with the assemblies currently on production, in case we needed to fix live bugs, and a v6.5 code branch in case we needed new code development to start that couldn’t wait for us to go live with 6.5. Branching definitely worked out for us at the end.

- If your database is anything like ours – very big and multilingual, sometimes it’s better to start with a clean v6.5 install and port things over slowly, but confidently. I tried an in-place upgrade about a year go. That was only going up 2 versions, from v6.1 to v6.3. That blew up in my face because of web.config typos. It’s definitely easier to start clean and go from there.

Conclusion

I truly hope these lessons will help you in your upgrade ventures by preparing you for the “gotchas” that could stand in your way. At the end of the day, it’s definitely worth it knowing that a) you are not obsolete anymore and running the latest and greatest (for the time being), b) you open yourself up to a full library of shared source modules and new features that weren’t available in the older versions. My favorite one – Sitecore Azure. By the way, I blogged about that experience as well, in a series titled “To The Cloud” here: http://blog.image0.com/tag/sitecore-azure-series/

Using Sitecore Publishing Pipeline to Refresh External CDN Cache

Posted on December 10, 2011 by Max

This is either one of those things you may never have to use, or you come across a requirement and realize this is exactly what you need and just didn’t know what it was called. I fell into the latter category when I needed to programmatically refresh images in my media library that were cached in my company’s content delivery syndicator – Akamai. Thankfully, @AlexShyba pointed me in the right direction.

Goal
Clear external image/CDN cache when a media item gets published.

Solution

The PublishProcessor object is located in the Sitecore.Publishing.Pipelines.Publish namespace. We will need to inherit from this object to your own class and override the Process(PublishContext) method with our own custom functionality.

The PublishContext that’s passed in by Sitecore’s publishing processor has a list of items being published. All we have to do is iterate through the list and use the Sitecore Item’s Path.IsMediaItem property to determine if it’s a media library item or not. If it is, we just pass it or its full path to a custom method and use whatever API is provided by your CDN of choice to force refresh on it.

This is all being done in a separate Visual Studio project, so when we are done, we get a nice assembly .dll file out of it that we attach to the publish processor in our web.config.

The Code

using Sitecore.Data;
using Sitecore.Data.Engines;
using Sitecore.Data.Managers;
using Sitecore.Diagnostics;
using Sitecore.Publishing.Pipelines.Publish;

namespace Sitecore.Custom
{
	public class CacheClearer : PublishProcessor
	{
		private readonly List _cacheQueue = new List();

		public override void Process(PublishContext context)
		{
			Assert.ArgumentNotNull(context, "context");
			ProcessPublishedItems(context);
		}

		protected virtual void ProcessPublishedItems(PublishContext context)
		{
			if (context == null || context.PublishOptions == null || context.PublishOptions.TargetDatabase == null)
			{
				Log.Error("Context and/or publish settings are null",this);
				return;
			}

			ProcessHistoryStorage(context.PublishOptions.TargetDatabase); //this updates the housekeeping fields

			Log.Debug("There are " + _cacheQueue.Count + " items in the cache queue");

			var mediaUrls = = new List<string>();

			foreach (var id in _cacheQueue)
			{
				if (context.PublishOptions.TargetDatabase.Items.GetItem(id) != null)
				{
					var item = context.PublishOptions.TargetDatabase.Items[id];
					if (item.Paths.IsMediaItem)
						mediaUrls.Add(akamaiPath);

				}

			}

			SomeCustomCDNProcessor.RefreshMediaUrls(mediaUrls);

			Log.Info("*** Processing cache clear for item: " + id, this);
		}

		private void ProcessHistoryStorage(Database database)
        {
            _cacheQueue.Clear();

            var utcNow = DateTime.UtcNow;

            // accessing the date of last operation
            var from = LastUpdateTime(database);

            //Log.Debug("Last Update Time: " + from);
            //Log.Debug("Database: " + database.Name);
            // get the history collection for the specified dates:
            var entrys = HistoryManager.GetHistory(database, from, utcNow);
            //Log.Debug("entry count: " +entrys.Count);
            if (entrys.Count > 0)
            {
                foreach (var entry in
                    entrys.Where(entry => !_cacheQueue.Contains(entry.ItemId) && entry.Category == HistoryCategory.Item))
                {
                    _cacheQueue.Add(entry.ItemId);
                    database.Properties[LastUpdate] = DateUtil.ToIsoDate(entry.Created, true);
                }
            }

            // writing back the date flag of our last operation
            database.Properties[LastUpdate] = DateUtil.ToIsoDate(utcNow, true);
        }

        protected DateTime LastUpdateTime(Database database)
        {
            var lastUpdate = database.Properties[LastUpdate];

            if (lastUpdate.Length > 0)
            {
                return DateUtil.ParseDateTime(lastUpdate, DateTime.MinValue);
            }

            return DateTime.MinValue;
        }

 }
}

The web.config changes are as follows:

<publish help="Processors should derive from Sitecore.Publishing.Pipelines.Publish.PublishProcessor">
        <processor type="Sitecore.Publishing.Pipelines.Publish.AddLanguagesToQueue, Sitecore.Kernel" />
        <processor type="Sitecore.Publishing.Pipelines.Publish.AddItemsToQueue, Sitecore.Kernel" />
        <processor type="Sitecore.Publishing.Pipelines.Publish.ProcessQueue, Sitecore.Kernel" />
        <processor type="Sitecore.Custom.CacheClearer,CustomCacheClearer" />
 </publish>

Sitecore Azure: Walkthrough – Installation to Deployment

Posted on July 13, 2011 by Max

My tweet from 5 days ago:

“hitting the green arrow to deploy content delivery environment in #Sitecore #Azure and actually seeing it work – brings a tear to my eye.”

You, too, can weep tears of joy after seeing exactly how magical this process is. I did bang my head against the wall a couple of times, but the wonderful support staff @ Sitecore, including Jesper Ravnsgaard, Product Manager for Sitecore, did help me reach the finish line.

First thing is first – you have to make sure you have met all the prerequisites and read the known issues from my previous post. The second thing you want to do is download the Windows Azure SDK from here. The third thing is download the latest version of Sitecore Azure from here, which is currently at version 1.0.4. If you read my previous post, you should have already sent out a request for and received a Sitecore Azure environment file. You cannot proceed without it. So don’t try. Seriously. Don’t.

Install The Certificate

You know the “10% inspiration/90% perspiration” shpiel? Getting the certificate to work correctly was the 90% perspiration part me. You have to follow each step verbatim, otherwise you will lose valuable time going through logs and emailing support. If you get stuck and this post does not help you, the latest version of the Troubleshooting Guide should help you pass this hurdle; otherwise, there is always Sitecore Support. My problem during this is that I was working on Sitecore Azure v1.0.4, while my guide was v1.0.3. Small detail that cost me 2 days. No biggie.

To install the certificate, there are a couple of steps:

Make sure you are on v1.0.4. You will have to generate the certificate as Administrator. The most full-proof way is via command prompt.
Open command prompt up as Administrator
cd to {your sitecore install directory}/sitecore/shell/Applications/azure. I know that the documentation states to do it from /sitecore/admin/azure, but I was getting System.IO.FileNotFound exceptions until Sitecore Support pointed out that for v1.0.4 you need to be doing this from the former directory.
Type MakeCert.bat to generate and install the certificate.
The certificate will need read rights by the account under which the App Pool is running. In IIS7, there is a user group called IIS_IUSRS. Make sure to grant it, along with NETWORK SERVICE read rights. You would do this in the Certificates Snap-In under mmc.exe. Screenshots and directions are located on page 5 of the Troubleshooting Guide.

Know Your Rights

The way Sitecore Azure deploys your files is by copying the all your web files from the website root ({sitecore install path}\{website path}) to a WebRole directory under the $data folder of your installation. Therefore, you will need to grant read access to the NETWORK SERVICE user and IIS_IUSRS user group to the entire website root directory and its subdirectory. In some cases, you may see that both will need write access to your web.config. You can skip that if you are reluctant to do that, but your log files will immediately tell you if there is a problem with updating the web.config.

“To The Cloud”

Before we can proceed, part of deploying to the cloud is creating the core and web databases in your Azure account. It won’t be able to do so until you have allowed it access via firewall rules. You will need add to the external IP of the server you are deploying this from into the firewall rules. You can do that by navigating to the “Databases” section of your Azure portal, selecting your database, and clicking Firewall Rules.

At this point, your certificate is installed and your permissions are set up, and your SQL Azurefirewall rules are set. You are ready for flight takeoff. You can now click the Sitecore “start” button and navigate to Azure. If this is your first time running it, it will ask you to upload the environment file you received from Sitecore. Next, if it doesn’t hang or give you any certificate errors, congratulations. You are quite good at following directions. If not, try again from the top.

Clicking the green button will start the process that will copy your website files to the data folder and deploy them to the cloud. This process will typically take 2-5 minutes. If you are at this point, you can breathe easy. The rest is a piece of cake. After the transitioning is complete, you will get a long URL where your app is hosted (something.cloudapp.net). If you are like me, you will probably click it before you even finish reading this sentence and find nothing. Why is this the case, you ask?

Your files are deployed, but your databases are blank. Luckily, it’s an easy step – we just publish the entire site to the azure target.

Note: I ran into some issues here. For some reason, when clicking “publish,” I got an error that the connection string for the azure target was missing. I contacted Sitecore Support and they said it may have been a fluke, but in case this happens to you, just insert it manually into your /App_Config/ConnectionStrings.config file. The exact connecting string can be found in Sitecore’s Content Manager under /System/Azure if you navigate down to the database.

After it publishes, you can click on the hostname in the top Windows Azure icon which will end with cloudapp.net.

That’s it. Quite an easy process if you follow the directions step by step. It gets a little more complicated if you have additional databases, which you will have to create through SQL Azure, but other than that, this is a very straight-forward approach.

Do not hesitate to reach out to me with any questions or comments about this.

Sitecore Azure: Installation Prerequisites & Known Issues

Posted on June 28, 2011 by Max

[UPDATE 7/13/2010]: Jesper Ravnsgaard, the product manager for Sitecore Azure, was kind enough to point out some corrections. They are below in blue next to the crossed-out text.

This post will cover the prerequisites and getting out the gate with Sitecore Azure. If you haven’t yet received your Azure account credentials, check my previous post.

Also, there are two things to bear in mind.

1) Sitecore Azure is its beginning developmental stages, so while it may work in most common scenarios, it will not work in all (see known issues below).

2) I am also new to Azure development, especially Sitecore Azure, so this is as much a learning experience for me as it is for you. If I stumble on anything contradictory in future posts, I will be sure to update any incorrect details.

Known Issues

Before you continue, it’s important to see if there are any deal-breakers. The following are some important considerations from the Sitecore Azure Troubleshooting Guide.

Sitecore Azure does not support file media. An example of this would be an app that lets a user upload a picture to the file system.
Cherry-picked file deployments to the content delivery environment are currently not doable. If you want to deploy even something as small as a CSS or a javascript, you will have to deploy the entire solution.
There is an update button. It looks for files that are different from the files deployed in the cloud and copies the delta. On the one hand, it doesn’t overwrite the entire solution, but on the other hand, it copies the entire delta, so if you wanted to deploy a newer CSS file, but not the newer javascript file, you would have to 1) backup the files you don’t want deployed, 2) restore them to the stable versions on production, 3) click the update button, and 4) restore your changed files.
For emergency file patching, you have RDP access to the instances.

The last prerequisite before the install is getting your Sitecore Azure Environment File. I am going to request mine as soon as this post is published. To get your environment file, email azure.accounts@sitecore.net with the following information:

The location, hostname, login and password for the Microsoft Azure DBServer(s).
Your Certificate file: /sitecore/admin/azure/certificate.cer file from Sitecore Azure installation.
Your Sitecore license file

I have yet to request and receive mine, but when I do, I’ll be sure to post the installation details with some screenshots and walk you through the installation. In the meantime, please refer to the following Sitecore documents for detailed information:

image0.com blog

development including .net, sharepoint, and sitecore CMS

Category Archives: Sitecore