Jon Gregory's Blog

Developer, Technical Architect, New Father , Lapsed Runner and Mountain Biker

C# Importance of Checking for NULL's

This week I was reviewing some C# code where the developer had used 'ToString()' throughout on properties without checking for nulls. 

The result of this was a log file full of 'System.NullReferenceException: Object reference not set to an instance of an object.'

And it was difficult to trace through where the error had occurred and the logic stopped executing.

So I put together a few simple samples of how to safely check for nulls and work with collections,  I can distribute them to the training developers and add hope to add more to this project over time. The code is  available here;


Some of the samples are below.

Examples Ways to handle null values

AS Operator

//https://msdn.microsoft.com/en-us/library/cscsdfbt.aspx

var asOperator = classWithNulls.IamNull as string;
Console.WriteLine("Null is returned by as with no exception : " + asOperator);

Convert Class

// https://msdn.microsoft.com/en-us/library/system.convert(v=vs.110).aspx

var convertToString = Convert.ToString(classWithNulls.IamNull);
Console.WriteLine("Convert Class returns empty string if null no exception : " + convertToString);

Coalesce Operator

//https://msdn.microsoft.com/en-GB/library/ms173224.aspx

var coalesce = classWithNulls.IamNull ?? "DefaultValueIfNull";
Console.WriteLine("DefaultValue is returned if null : " + coalesce);

Conditional Operator

//https://msdn.microsoft.com/en-gb/library/ty67wk28.aspx

// ReSharper disable once MergeConditionalExpression
var conditional = classWithNulls.IamNull != null
    ? classWithNulls.IamNull.ToString()
    : "DefaultValueIfNull";

Console.WriteLine("DefaultValue is returned if null : " + conditional);

// C# 6 and later null propagation operator where the null value is passed up the chain without an exception
//https://msdn.microsoft.com/en-GB/library/dn986595.aspx

var conditionalCSharp6 = classWithNulls.IamNull?.ToString();

Console.WriteLine("Null is propagated and returned without an Exception" + conditionalCSharp6);

// This can be combined with Coalesce to remove the null and provide a default value
var conditionalCSharp6DefaultValue = classWithNulls.IamNull?.ToString() ?? "DefaultValueIfNull";
Console.WriteLine("Default Value Returned without an Exception" + conditionalCSharp6DefaultValue);

Extension Method


public static class Extension
    {
        public static string ToStringOrEmpty(this Object value)
        {
            return value == null ? "" : value.ToString();
        }
    }


// A custom extension method attached to object to handle null values, see file ToStringOrEmpty.cs
// https://msdn.microsoft.com/en-gb/library/bb383977.aspx

var extensionMethod = classWithNulls.IamNull.ToStringOrEmpty();
Console.WriteLine("String Empty is returned by as with no exception : " + extensionMethod);



SSL V 3.0 - Disable Poodle Threats on IIS


On a recent PEN test the SSL V3 POODLE bug was picked up on the server http://www.symantec.com/connect/blogs/ssl-30-vulnerability-poodle-bug-aka-poodlebleed

To test a website this handy page reports on any url visible to the browser https://cryptoreport.thawte.com/checker/views/certCheck.jsp

I assumed this would be a setting in IIS but the advice from Microsoft is a quick registry edit - detailed in the 'Disable SSL 3.0 Server Software' section in this article https://technet.microsoft.com/library/security/3009008.aspx

Disable SSL 3.0 in Windows For Server Software

You can disable support for the SSL 3.0 protocol on Windows by following these steps:

  1. Click Start, click Run, type regedt32 or type regedit, and then click OK.
  2. In Registry Editor, locate the following registry key:
    HKey_Local_Machine\System\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0\Server
    

    Note If the complete registry key path does not exist, you can create it by expanding the available keys and using the New -> Key option from the Edit menu.

  3. On the Edit menu, click Add Value.
  4. In the Data Type list, click DWORD.
  5. In the Value Name box, type Enabled, and then click OK

    Note If this value is present, double-click the value to edit its current value.

  6. In the Edit DWORD (32-bit) Value dialog box, type 0 .
  7. Click OK. Restart the computer.

 

Note This workaround will disable SSL 3.0 for all server software installed on a system, including IIS.









Service Design and Implementation - Checklist

I have always been a fan of checklists, but not always great at putting them together! A useful tip I saw on a course a few years ago was to create checklist for common software architecture tasks. 

These are great for those time critical situations where a decision is needed quickly, and for adding Gotchas and lessons learned too for next time...

Here is my first checklist I use when designing and building a new web service, this has been put together from a variety of resources and my own experience. Its not finished and will hopefully get added to with each new service built.

Service Contracts

  • Inbound and Outbound Data Structure
  • Format (JSON, JSON-LD, XML..)
  • Service Based Contract (Service owns the contract and can change)
  • Consumer-Driven Contract (Collaborative, needs of the consumer taken into account)
  • Homogenous Contract Versioning (Version Number, new version of same contract/method)
  • Heterogeneous Contract Versioning (Support multiple contracts, more consumer driven)
  • Backward Compatibility - see above
  • Service Consumer Communication Strategy (Notify Changes)

Data / Payload

  • Frequency
  • Size - Divide up?
  • Images - separate ftp?
  • Design Pattern (Command, Document , Event)
  • Relationships
  • Pagination 
  • Delete Strategy (Mark for deletion? , range of deletion)
  • Dates - Local or UTC
  • DTO layer to control exposure

Availability

  • Availability  - ability to call
  • Responsiveness - timely response
  • Timeout strategy - load test to find time under load and double
  • Circuit Breaker Pattern - no response trips the circuit preventing others to have to wait for time outs
  • Smart timeouts - vary in load and request us code to manage

Exception Strategy

  • Monitoring
  • Logging
  • Alerts
  • Retries
  • Availability Monitoring
  • Latency and Response Times
  • Method retry in case of failure?
  • Critical and Non-Critical dependancies  - can service proceed with non critical

Implementation

  • Business Logic and Service Logic separated
  • Translation Layer separated 
  • Caching strategy
  • Consistent and Intuitive naming
  • Correct HTTP Response code
  • Error Codes
  • Error Messages - add extra information?
  • Versioning implantation (url, header…)

Security

  • Encryption Strategy 
  • Authentication - (Username/PWD, Claims Based, etc)
  • Authorisation
  • Separate security service required?
  • Limited exposure of implementation detail
  • Access restriction - calls per second 
  • DDOS prevention
  • Unique IDs over Auto Increment Integers

Transactions

  • Atomicity, Consistency, Isolation, Durability (ACID)
  • Basic Availability, Soft State, Eventual Consistency (BASE)
  • Data Consistency 

Complexity

  • Does the problem require the complexity of SOA and MicroServices
  • Will a simple asmx service suffice at the start

Taxonomy

Classification within an architecture

    • Service Type - At Architecture Pattern Level
    • Business Area - At Implementation Level

Microservices

    • Functional Services - Business Operations
    • Non-Functional - authentication / logging etc

Ownership

  • Which organisation is responsible for change 
  • Need to understand the groups to coordinate with to deliver the service   business, enterprise, application, infrastructure - More of an impact for SOA

Granularity

  • SOA Larger course grained
  • MicroServices smaller on function services
  • Effects performance and Transaction Management
  • ACID difficult over fine grained services
  • Start with small number of large services and decompose 

Service Patterns

  • SOA
  • MicroServices

Orchestration and Coordination

  • Orchestration - Coordination of multiple services via a centralised mediator
  • Choreography  - Coordination of multiple services via without a centralised mediator - Inter Service Communication or Service Chaining (Micro service Preferred)

Remote Access Protocol

  • REST
  • SOAP
  • AMQP
  • Java Message Service (JMS)
  • MSMQ
  • Remote Method Invocation (RMI)

Application Scope

  • SOA - large complex enterprise wide
  • Microservices - smaller , well-portioned web based systems
  • Heterogeneous Interoperability - interactions with systems written in different languages

Testing

  • Data Seeding - Automated Seeding Tool
  • Test as early as possible
  • BDD tool for integration testing
  • Debugging tools

Documentation

  • Types
  • Access
  • Discoverable - landing resources?
  • Sample Code
  • Swagger, API Blueprint

ASP.NET Web Forms - Intermittent View State Error


On a web forms application on load balanced environment I recently had to investigate some errors for invalid viewstate. It was an intermittent error which only occurred when someone important used the site.

This became high profile quickly and very annoying, it has been a long time since I had to deal with view state thanks to MVC and it was difficult to reproduce. Eventually we found that clearing all caches and browser history allowed the error to be recreated.

The error message was a value cannot be null error;

System.Web.HttpException: Value cannot be null.
Parameter name: inputString [ArgumentNullException: Value cannot be null.Parameter name: inputString]
at System.Web.UI.ObjectStateFormatter.Deserialize(String inputString, Purpose purpose)
at System.Web.UI.Util.DeserializeWithAssert(IStateFormatter2 formatter, String serializedState, Purpose purpose)
at System.Web.UI.HiddenFieldPageStatePersister.Load()
[ViewStateException: Invalid viewstate. Client IP: 86.131.46.52 Port: 49531 Referer: https://*********.com/checkout/payment Path: *******.aspx User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko ViewState: ] [HttpException: The state information is invalid for this page and might be corrupted.]

The machinekey was in the web.config for both servers and the load balancing set up correctly. Looking at New Relic there was more viewstate errors which didn't hit the application so were not logged.

The most likely solution found when googling was to move the machine key to the machine.config file at the server level C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config

The site web.config had the machine key removed and there is a web.config in the same directory as the machine.config , in one server this had the machine key and on another it didn't. So it was removed from all.

This url has some useful information about the scope of the configuration files


ASP.NET 4.5 introduced cryptographic improvements, when the configuration was set wrong these were picked up and the following error was received.

System.Configuration.ConfigurationErrorsException: When using <machineKey compatibilityMode="Framework45" /> or the MachineKey.Protect and MachineKey.Unprotect APIs, the 'validation' attribute must be one of these values: SHA1, HMACSHA256, HMACSHA384, HMACSHA512, or alg:[KeyedHashAlgorithm]. (C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\web.config line 31)

Explicitly setting the validation key' validation="SHA1" ' to ensure the correct encryption  setting  was picked up solved this.

There is more information about the improvements here;


But after all that reconfiguration  the error popped up once more in New Relic!

After some more investigation we found that the viewstate had been moved to the bottom of the page for SEO, on the page receiving the error it was possible to submit the page before it had completed loading and then error then occured. Moving the viewstate back to the top seems to have cured it but we are keeping a close eye on New Relic

Sql Server Management Studio Results to Excel

I need to do some quick analysis on a error log table, so was running a grouping query on the log table.

To get the results into excel I wanted to cut and paste the results from SSMS query window into excel.

Annoyingly the .Net Exceptions where going into Excel on new rows which made it unreadable. 


With SQL 2012 the line feeds and carriage returns are preseved  and needed to be filtered out in the query

SELECT COUNT(*) as count, replace(replace(Message, char(10), ''), char(13), '') as 'Message' FROM Log

Session State being lost in Web Farm

Investigating and issue with session data being lost on a load balance ASP.Net website the other day, users we're losing items from a basket when bouncing between servers.

These issues have always been difficult to investigate but it turned out that when setting up the web servers there was multiple sites in IIS. These has been created in different order so had ended up with different site ids. A simple error but the symptoms were dramatic for users and difficult to recreate and trace.

This article details the issues https://support.microsoft.com/en-us/kb/325056 , although I did not have to use the described resolution.

I found that I could just change the site id through IIS, if a another site had the required id I renamed that until both web servers were is sync.

In IIS select the site and advanced settings; the edit the id to the required value




Hyper-V Virtual Machine Copy

I needed to evaluate a several database tools against an application I was writing,  so it made sense to create VM with the application and then copy it. Looking into Hyper-V  there was no option to clone a Virtual Machine without puchasing a tool. 

Looking at google there were two options;

  1. Copy the virtual hard disk in file explorer and then create a new VM using the Hyper-V wizard and point to the copied VHD when setting up the disk
  2. Export the VM and then import as a new VM using the import export tool.

The following steps are how to do option 2 , note that this operation can only be performed once for each export. The export files can only be imported once, if you need to do multiple copies you will need to do another export for each import.

In the Hyper-V Manager

1 - Chose Export for the right hand side menu



2 - Select a location for the export files to go, this is a temporary location as once imported the directory will remain but cannot be used again.




3 - The export can take a while .....



Importing the exported Virtual Machine

1 - In the Hyper-V Manager select 'Import a Virtual Machine'


2 - Select the folder where the exported Virtual Machine files were placed, selecting the top level folder is ok


3 - Select the location for the new Virtual Machine to be created, note you will need to change these values as they will be the same as the original machine and it will attempt to overwrite.



4 - It is important to select 'Copy the virtual machine' otherwise it will overwrite the original machine, the import can take a while.



5 - The imported VM will appear to have a duplicate name, this can be changed by right clicking on the name and changing to the required name


Once complete the VM can be started and used and is an exact copy of the original, this was ideal for the scenario as the evaluation was using trail period software and the VM's were only required for a few days.


Getting Started with Xamarin - Setup


I wanted to learn Xamarin for a while and with recent purchase by Microsoft and inclusion in Visual Studio now was the time.

This blog is not meant to reinvent the wheel and there are many excellent guides out there on how to get started. This is my experience following the MSDN setup documentation  and the tweaks I had to make. As much of personal record as a blog post.

Issues I had along the way following the setup; 

Has errors  with the Andriod SDK after the initial install trying to create a new Android app, followed this blog to resolve. Had to launch the SDK Manager after install and run  to install the full SDK components



Needed to enable Hyper-V on Windows 10, it was showing as enabled but it wouldn't work on my PC . Followed the blog post below and had to disable and reenable through windows features which meant a few restarts!


To check everything had installed properly I ran through these verification steps which really helped, I found I needed to install the Windows Phone Emulator and when connecting to the Mac had to set remote access to all users to get working


Hurrah all the projects all run!

Next step is to create a simple app with shared code approach  https://msdn.microsoft.com/en-us/library/dn879698.aspx
 

Sql Server Management Studio Freezing On Start Up

For a few weeks I had a really annoying issue with Sql Server Management Studio freezing on start up, this happened a few times at crucial moments and caused me delays.

I tried several re-installs and repairs and even the 2016 RC version, but it would just keep happening randomly.

Until I found his blog and following the second option to clear the profile information.


An bingo it never happened again, turns out the profile data is not removed on a uninstall.



Migrating Blog to SQL Azure Database

This is post is about my experiences migrating my blog to a SQL Azure Database.

When I started with BlogEngine.Net I used the put of the box persistence which is XML file based in the App_Data folder.

I wanted to use a SQL Azure Database and migrate the data across, I found the blog post below with instructions.


I took the database script from the setup folder in the BlogEngine.NET file system and ran it against the New SQL Azure DB using SQL Server Managment Studio.

This gave me most of what was required but I had to change the BlogMigration.aspx page to remove the '1' from the master page name;

<%@ Page Language="C#" MasterPageFile="~/admin/admin1.master" AutoEventWireup="true" CodeFile="BlogMigration.aspx.cs" Inherits="admin_Pages_BlogMigration" Title="Blog Migration" %>

I still had some compile errors , the following blog post describes the changes required to the BlogMigration.aspx.cs code file, and how to set the blog guid id.


I had to check that the firewall setting were correct for my new SQL Azure DB and that it allowed connections from Azure Web Servers. This post describes how to do that


The users were missing from the database , so I took the encrypted passwords out of the \App_Data\Users.xml and put into the database. This link shows where to find the user information  in the XML and the database.


I also found I had to take the blog guid and change the user and role table to use the imported blog guid, they were all set up with the guid from the set up script.

A few niggle bit was very easy to do in the end thanks to the migration tool and above blog posts.