Blog Rebuild with HTTPS all the time

 Ok so it was time for the blog to receive some TLC, I am running BlogEngine.Net version 2.9 as an azure web app with an azure SQL database. Version 3.3 of BlogEngine.Net was available. There has been a lot of improvements to the user interface and editor. I was also running a multi user and multi blog set up which made the database upgrades trickier. So I opted for a full clean install and then to migrate the data over. I am not a profilic author so there was not too much to migrate.

Also with the push towards HTTPS for all pages and chrome marking pages insecure in 2017, https://security.googleblog.com/2016/09/moving-towards-more-secure-web.html I wanted to upgrade to HTTPS. The best way to do this was to use a Lets Encrypt certificate which is a free automated certificate authority.

Clean Installation of BlogEngine.Net

This was the easiest part simply download the zip file from http://blogengine.codeplex.com/releases/view/621156, then extract the files. The web.config needed to be updated to use the DbBlogProvider and a connection string added. 

    <blogProvider defaultProvider="DbBlogProvider" fileStoreProvider="XmlBlogProvider">
      <providers>
        <add description="Xml Blog Provider" name="XmlBlogProvider" type="BlogEngine.Core.Providers.XmlBlogProvider, BlogEngine.Core" />
		<add connectionStringName="BlogEngine" description="Sql Database Provider" name="DbBlogProvider" type="BlogEngine.Core.Providers.DbBlogProvider, BlogEngine.Core" />
      </providers>
    </blogProvider>

 

The Membership Provider and Role Provider also needed to be updated to use the database although this is less important as I am sticking to single user mode.

 <membership defaultProvider="DbMembershipProvider">
      <providers>
        <clear />
        <add name="XmlMembershipProvider" type="BlogEngine.Core.Providers.XmlMembershipProvider, BlogEngine.Core" description="XML membership provider" passwordFormat="Hashed" />
		<add name="DbMembershipProvider" type="BlogEngine.Core.Providers.DbMembershipProvider, BlogEngine.Core" passwordFormat="Hashed" connectionStringName="BlogEngine" />
      </providers>
    </membership>
    <roleManager defaultProvider="DbRoleProvider" enabled="true" cacheRolesInCookie="false">
      <providers>
        <clear />
        <add name="XmlRoleProvider" type="BlogEngine.Core.Providers.XmlRoleProvider, BlogEngine.Core" description="XML role provider" />
		<add name="SqlRoleProvider" type="System.Web.Security.SqlRoleProvider" connectionStringName="BlogEngine" applicationName="BlogEngine" />
        <add name="DbRoleProvider" type="BlogEngine.Core.Providers.DbRoleProvider, BlogEngine.Core" connectionStringName="BlogEngine" />
      </providers>
    </roleManager>

 

Once the blog was started up the admin section had a link to update to 3.3.5, which was upgraded automatically from the admin section without me having to touch a file. Very slick.

Adding LetsEncrypt

I have been wanting to try LetsEncrypt out for a while and this was ideal for my blog, I wasn't sure how to get a certificate onto a Azure WebApp using a custom domain. After some googling this Azure extension stood out a mile https://github.com/sjkp/letsencrypt-siteextension. Not only does it provide a UI for adding the certificate it also sets up a web job to refresh them when they expire. The LetsEncrypt certificates expire after 90 days , this is for security and to encourage automation to renew the licences. LetsEncrypt Expiration Notes

I found this great detailed blog post  which explained a lot of the reasons for the different steps, however the 'Register a Service Principal' in powershell looked a bit tricker than the time I had available so I jumped back over to the WIKI Documenation for the extension and that covered how to do it in the Azure Control Panel  

I found the extension failed on the last step the first time it ran, then I ran again and it worked. Wasn't able to find out why and may have just been a azure refresh speed on one of the set up steps

 

Add a https redirect (Skipped down to the enforce HTTPS section)

Again a nice easy one, just need to add the redirect rule into the web.config this Blog Post on Azure covers how to add a certificate to a custom domain, as I had done this with the Azure Extension I just needed to skip to the 'Enforce HTTPS section'

Adding the following section in the <system.webserver> section of the web.config moves all sessions to HTTPS

 <rewrite>
      <rules>
        <!-- BEGIN rule TAG FOR HTTPS REDIRECT -->
        <rule name="Force HTTPS" enabled="true">
          <match url="(.*)" ignoreCase="false" />
          <conditions>
            <add input="{HTTPS}" pattern="off" />
          </conditions>
          <action type="Redirect" url="https://{HTTP_HOST}/{R:1}" appendQueryString="true" redirectType="Permanent" />
        </rule>
        <!-- END rule TAG FOR HTTPS REDIRECT -->
      </rules>
    </rewrite>

 

Migrate old data 

 The final step was to migrate the data over from the old blog, I had created a new SQL Azure Database so just needed to take from three tables. be_Posts, be_Categories and be_PostCategory. I wrote the slightly funky query to generate a formatted insert statement and ran in SSMS with a results to text. Then moved the query over to the new database. It worked for all but three posts where the post data exceeded the limit in the results to text of SSMS. As there were only three I moved those posts manually. Not the cleanest of solutions I admit but I wanted a fresh install of the blog software and to make future upgrades easier by using the default blog guid as the blog id.

SELECT
'INSERT INTO [dbo].[be_Posts]
           ([BlogID]
           ,[PostID]
           ,[Title]
           ,[Description]
           ,[PostContent]
           ,[DateCreated]
           ,[DateModified]
           ,[Author]
           ,[IsPublished]
           ,[IsCommentEnabled]
           ,[Raters]
           ,[Rating]
           ,[Slug]
           ,[IsDeleted])
     VALUES
	 (',
      char(39) + '27604F05-86AD-47EF-9E05-950BB762570C' + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + convert(varchar(50),[PostID]) + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + [Title] + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + [Description] + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + '*' + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + convert(varchar(50),[DateCreated]) + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + convert(varchar(50),[DateModified]) + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + [Author] + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + convert(varchar(50),[IsPublished]) + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + convert(varchar(50),[IsCommentEnabled]) + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + convert(varchar(50),[Raters]) + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + convert(varchar(50),[Rating]) + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + [Slug] + char(39) + CHAR(13) + CHAR(10)
      ,',' + char(39) + convert(varchar(50),[IsDeleted]) + char(39) + CHAR(13) + CHAR(10)
	  ,')'
  FROM [blogjongregorydb].[dbo].[be_Posts]
GO

 

 Conclusion

Writing this first post has been so much easier, the editor in 3.3.5 is much better and the ability to add links is far quicker. The code window has more formatting and is easier to edit, and code snippets can be added in directly. Although I still prefer the formatting of http://hilite.me/ .

Adding the LetsEncrypt extension and certificate was really convenient and just need to check the renewal of the certificate in 90 days time. 

But the blog and my personal CV website is marked as secure and I feel confident using this approach on other websites.

C# Importance of Checking for NULLs

This week I was reviewing some C# code where the developer had used 'ToString()' throughout on properties without checking for nulls. 

 
The result of this was a log file full of 'System.NullReferenceException: Object reference not set to an instance of an object.'
 
And it was difficult to trace through where the error had occurred and the logic stopped executing.
 
So I put together a few simple samples of how to safely check for nulls and work with collections,  I can distribute them to the training developers and add hope to add more to this project over time. The code is  available here;
 
 
Some of the samples are below.
 
Examples Ways to handle null values
 
AS Operator
 
//https://msdn.microsoft.com/en-us/library/cscsdfbt.aspx

var asOperator = classWithNulls.IamNull as string;
Console.WriteLine("Null is returned by as with no exception : " + asOperator);
 
Convert Class
 
// https://msdn.microsoft.com/en-us/library/system.convert(v=vs.110).aspx

var convertToString = Convert.ToString(classWithNulls.IamNull);
Console.WriteLine("Convert Class returns empty string if null no exception : " + convertToString);
 
Coalesce Operator
 
//https://msdn.microsoft.com/en-GB/library/ms173224.aspx

var coalesce = classWithNulls.IamNull ?? "DefaultValueIfNull";
Console.WriteLine("DefaultValue is returned if null : " + coalesce);
 
Conditional Operator
 
//https://msdn.microsoft.com/en-gb/library/ty67wk28.aspx

// ReSharper disable once MergeConditionalExpression
var conditional = classWithNulls.IamNull != null
    ? classWithNulls.IamNull.ToString()
    : "DefaultValueIfNull";

Console.WriteLine("DefaultValue is returned if null : " + conditional);

// C# 6 and later null propagation operator where the null value is passed up the chain without an exception
//https://msdn.microsoft.com/en-GB/library/dn986595.aspx

var conditionalCSharp6 = classWithNulls.IamNull?.ToString();

Console.WriteLine("Null is propagated and returned without an Exception" + conditionalCSharp6);

// This can be combined with Coalesce to remove the null and provide a default value
var conditionalCSharp6DefaultValue = classWithNulls.IamNull?.ToString() ?? "DefaultValueIfNull";
Console.WriteLine("Default Value Returned without an Exception" + conditionalCSharp6DefaultValue);
 
Extension Method
 
 
public static class Extension
    {
        public static string ToStringOrEmpty(this Object value)
        {
            return value == null ? "" : value.ToString();
        }
    }


// A custom extension method attached to object to handle null values, see file ToStringOrEmpty.cs
// https://msdn.microsoft.com/en-gb/library/bb383977.aspx

var extensionMethod = classWithNulls.IamNull.ToStringOrEmpty();
Console.WriteLine("String Empty is returned by as with no exception : " + extensionMethod);
 
 
 

SSL V 3.0 - Disable Poodle Threats on IIS


On a recent PEN test the SSL V3 POODLE bug was picked up on the server http://www.symantec.com/connect/blogs/ssl-30-vulnerability-poodle-bug-aka-poodlebleed

To test a website this handy page reports on any url visible to the browser https://cryptoreport.thawte.com/checker/views/certCheck.jsp

I assumed this would be a setting in IIS but the advice from Microsoft is a quick registry edit - detailed in the ''Disable SSL 3.0 Server Software'' section in this article https://technet.microsoft.com/library/security/3009008.aspx

Disable SSL 3.0 in Windows For Server Software

You can disable support for the SSL 3.0 protocol on Windows by following these steps:

  1. Click Start, click Run, type regedt32 or type regedit, and then click OK.
  2. In Registry Editor, locate the following registry key:
    HKey_Local_Machine\System\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0\Server
    

    Note If the complete registry key path does not exist, you can create it by expanding the available keys and using the New -> Key option from the Edit menu.

  3. On the Edit menu, click Add Value.
  4. In the Data Type list, click DWORD.
  5. In the Value Name box, type Enabled, and then click OK

    Note If this value is present, double-click the value to edit its current value.

  6. In the Edit DWORD (32-bit) Value dialog box, type 0 .
  7. Click OK. Restart the computer.

 

Note This workaround will disable SSL 3.0 for all server software installed on a system, including IIS.









ASP.NET Web Forms - Intermittent View State Error


On a web forms application on load balanced environment I recently had to investigate some errors for invalid viewstate. It was an intermittent error which only occurred when someone important used the site.

This became high profile quickly and very annoying, it has been a long time since I had to deal with view state thanks to MVC and it was difficult to reproduce. Eventually we found that clearing all caches and browser history allowed the error to be recreated.

The error message was a value cannot be null error;

System.Web.HttpException: Value cannot be null.
Parameter name: inputString [ArgumentNullException: Value cannot be null.Parameter name: inputString]
at System.Web.UI.ObjectStateFormatter.Deserialize(String inputString, Purpose purpose)
at System.Web.UI.Util.DeserializeWithAssert(IStateFormatter2 formatter, String serializedState, Purpose purpose)
at System.Web.UI.HiddenFieldPageStatePersister.Load()
[ViewStateException: Invalid viewstate. Client IP: 86.131.46.52 Port: 49531 Referer: https://*********.com/checkout/payment Path: *******.aspx User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko ViewState: ] [HttpException: The state information is invalid for this page and might be corrupted.]

The machinekey was in the web.config for both servers and the load balancing set up correctly. Looking at New Relic there was more viewstate errors which didn''t hit the application so were not logged.

The most likely solution found when googling was to move the machine key to the machine.config file at the server level C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config

The site web.config had the machine key removed and there is a web.config in the same directory as the machine.config , in one server this had the machine key and on another it didn''t. So it was removed from all.

This url has some useful information about the scope of the configuration files


ASP.NET 4.5 introduced cryptographic improvements, when the configuration was set wrong these were picked up and the following error was received.

System.Configuration.ConfigurationErrorsException: When using <machineKey compatibilityMode="Framework45" /> or the MachineKey.Protect and MachineKey.Unprotect APIs, the ''validation'' attribute must be one of these values: SHA1, HMACSHA256, HMACSHA384, HMACSHA512, or alg:[KeyedHashAlgorithm]. (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\web.config line 31)

Explicitly setting the validation key'' validation="SHA1" '' to ensure the correct encryption  setting  was picked up solved this.

There is more information about the improvements here;


But after all that reconfiguration  the error popped up once more in New Relic!

After some more investigation we found that the viewstate had been moved to the bottom of the page for SEO, on the page receiving the error it was possible to submit the page before it had completed loading and then error then occured. Moving the viewstate back to the top seems to have cured it but we are keeping a close eye on New Relic

Sql Server Management Studio Results to Excel

I need to do some quick analysis on a error log table, so was running a grouping query on the log table.

To get the results into excel I wanted to cut and paste the results from SSMS query window into excel.

Annoyingly the .Net Exceptions where going into Excel on new rows which made it unreadable. 


With SQL 2012 the line feeds and carriage returns are preseved  and needed to be filtered out in the query

SELECT COUNT(*) as count, replace(replace(Message, char(10), ''''), char(13), '''') as ''Message'' FROM Log

Session State being lost in Web Farm

Investigating and issue with session data being lost on a load balance ASP.Net website the other day, users we''re losing items from a basket when bouncing between servers.

These issues have always been difficult to investigate but it turned out that when setting up the web servers there was multiple sites in IIS. These has been created in different order so had ended up with different site ids. A simple error but the symptoms were dramatic for users and difficult to recreate and trace.

This article details the issues https://support.microsoft.com/en-us/kb/325056 , although I did not have to use the described resolution.

I found that I could just change the site id through IIS, if a another site had the required id I renamed that until both web servers were is sync.

In IIS select the site and advanced settings; the edit the id to the required value




Hyper-V Virtual Machine Copy

I needed to evaluate a several database tools against an application I was writing,  so it made sense to create VM with the application and then copy it. Looking into Hyper-V  there was no option to clone a Virtual Machine without puchasing a tool. 

Looking at google there were two options;

  1. Copy the virtual hard disk in file explorer and then create a new VM using the Hyper-V wizard and point to the copied VHD when setting up the disk
  2. Export the VM and then import as a new VM using the import export tool.

The following steps are how to do option 2 , note that this operation can only be performed once for each export. The export files can only be imported once, if you need to do multiple copies you will need to do another export for each import.

In the Hyper-V Manager

1 - Chose Export for the right hand side menu



2 - Select a location for the export files to go, this is a temporary location as once imported the directory will remain but cannot be used again.




3 - The export can take a while .....



Importing the exported Virtual Machine

1 - In the Hyper-V Manager select ''Import a Virtual Machine''


2 - Select the folder where the exported Virtual Machine files were placed, selecting the top level folder is ok


3 - Select the location for the new Virtual Machine to be created, note you will need to change these values as they will be the same as the original machine and it will attempt to overwrite.



4 - It is important to select ''Copy the virtual machine'' otherwise it will overwrite the original machine, the import can take a while.



5 - The imported VM will appear to have a duplicate name, this can be changed by right clicking on the name and changing to the required name


Once complete the VM can be started and used and is an exact copy of the original, this was ideal for the scenario as the evaluation was using trail period software and the VM''s were only required for a few days.


Getting Started with Xamarin - Setup


I wanted to learn Xamarin for a while and with recent purchase by Microsoft and inclusion in Visual Studio now was the time.

This blog is not meant to reinvent the wheel and there are many excellent guides out there on how to get started. This is my experience following the MSDN setup documentation  and the tweaks I had to make. As much of personal record as a blog post.

Issues I had along the way following the setup; 

Has errors  with the Andriod SDK after the initial install trying to create a new Android app, followed this blog to resolve. Had to launch the SDK Manager after install and run  to install the full SDK components



Needed to enable Hyper-V on Windows 10, it was showing as enabled but it wouldn''t work on my PC . Followed the blog post below and had to disable and reenable through windows features which meant a few restarts!


To check everything had installed properly I ran through these verification steps which really helped, I found I needed to install the Windows Phone Emulator and when connecting to the Mac had to set remote access to all users to get working


Hurrah all the projects all run!

Next step is to create a simple app with shared code approach  https://msdn.microsoft.com/en-us/library/dn879698.aspx
 

Sql Server Management Studio Freezing On Start Up

For a few weeks I had a really annoying issue with Sql Server Management Studio freezing on start up, this happened a few times at crucial moments and caused me delays.

I tried several re-installs and repairs and even the 2016 RC version, but it would just keep happening randomly.

Until I found his blog and following the second option to clear the profile information.


An bingo it never happened again, turns out the profile data is not removed on a uninstall.



Migrating Blog to SQL Azure Database

This is post is about my experiences migrating my blog to a SQL Azure Database.

When I started with BlogEngine.Net I used the put of the box persistence which is XML file based in the App_Data folder.

I wanted to use a SQL Azure Database and migrate the data across, I found the blog post below with instructions.


I took the database script from the setup folder in the BlogEngine.NET file system and ran it against the New SQL Azure DB using SQL Server Managment Studio.

This gave me most of what was required but I had to change the BlogMigration.aspx page to remove the ''1'' from the master page name;

<%@ Page Language="C#" MasterPageFile="~/admin/admin1.master" AutoEventWireup="true" CodeFile="BlogMigration.aspx.cs" Inherits="admin_Pages_BlogMigration" Title="Blog Migration" %>

I still had some compile errors , the following blog post describes the changes required to the BlogMigration.aspx.cs code file, and how to set the blog guid id.


I had to check that the firewall setting were correct for my new SQL Azure DB and that it allowed connections from Azure Web Servers. This post describes how to do that


The users were missing from the database , so I took the encrypted passwords out of the \App_Data\Users.xml and put into the database. This link shows where to find the user information  in the XML and the database.


I also found I had to take the blog guid and change the user and role table to use the imported blog guid, they were all set up with the guid from the set up script.

A few niggle bit was very easy to do in the end thanks to the migration tool and above blog posts.