Using Chef to edit app.config and web.config during a run

I’ve run into an issue with our Chef automation where i’ve found it extremely tricky to update app/web.config files, so i’ve put together the dotnetconfig cookbook to take care of this.

Code available here:


To use it, first download it from the Chef supermarket and unzip it to your cookbooks folder :-

knife supermarket download dotnetconfig --file dotnetconfig.tar.gz
tar -zxvf dotnetconfig.tar.gz

(you will also need to do this with nokogiri, on which it is dependent)

Use it within your own cookbooks by adding the following line to your cookbook’s metadata.rb.

depends         'dotnetconfig'

Within a recipe, add the following lines to include the recipe and import the DotNetConfig module.

include_recipe 'dotnetconfig'
::Chef::Recipe.send(:include, DotNetConfig)

Using the library

config_set_app_setting(document, 'fix-gateways', 'Replaced')
config_set_connection_string(document, 'Database1', 'Database=Partial;User Id=Replacement;Password=PasswordChanged;CommandTimeout=30;MinPoolSize=0')

Full usage instructions are available here.

Chef Knife-EC2 Windows Timeouts

We found we were getting some issues with the Knife-EC2 command KNIFE EC2 SERVER CREATE timing out when creating Windows notes on EC2.

The error we were getting was this :-

Fog::Compute::AWS::Error: RequestLimitExceeded

It seems to be due to the check_windows_password_available method having its “sleep” in the incorrect place. I’ve resolved this by editing the code for KNIFE-EC2 and moving the sleep to the top of the function.

def check_windows_password_available(server_id)
sleep 10
#Added this at the start of the loop as it doesn't fire if the return false is hit
response = connection.get_password_data(server_id)
if not response.body["passwordData"]
return false

I recompiled the GEM and installed it, and it’s now working without issues. Have posted this as a bug but not sure anyone believes me.

Chef – Rebooting a node without breaking the run

We’ve found ourselves in a situation where we need to reboot a Windows node after joining the domain, and this was causing the Chef run to fail as the node can’t be contacted. To get around this, simply execute a shutdown asynchronously with the shutdown command.

#Set up reboot after 30 seconds
execute "chef-client" do
command 'shutdown /r /t 30'
action :run

This sets up a reboot in 30 seconds, the Chef run will finish (as long as this is the last thing left) and the machine will come up on the domain.

Connecting Amazon VPCs in different regions using OpenSwan VPN

I’ve recently been working with Chef automation, and have needed to create two VPC in different regions (for risk purposes). AWS doesn’t support VPC pairing across region yet, so we’ve had to find a workaround.

The simplest solution has been to create two Linux instances (Ubuntu 14.04) in each region, and configure a VPN tunnel. Here are the steps we took, and some configuration gotchas to help anyone who might have this issue along the way.

Create VPCs with non-overlapping subnets

I’ve created two, one with the range and the other with


Create Linux instances in each VPC to serve as our VPN tunnel.

I’ve called mine OpenSwanA and OpenSwanB for the purpose of this tutorial.

Give them BOTH an elastic public IP.

OpenSwanA (

OpenSwanB (


Setup AWS instances (disable source destination check)

Very important! Go to each OpenSWAN instance and go to Actions -> Change Source/Dest. Check and set to DISABLE on both instances, or they won’t be able to see anything other than each other.


Configure route tables to send traffic for “VPC2” to the Linux VPN instance 

VPC1 -> local -> Internet Gateway (igw_xxxx) -> remote (select your OpenSwanB AWS Linux, start typing its Id)


VPC2 -> local -> Internet Gateway (igw_xxxx) -> remote (select your OpenSwanA AWS Linux, start typing its Id)


Now, any traffic designated for the 172.32.x for any instance in VPC1 will go out to the Linux instance, and vice-versa. Now, we just need to configure the VPN tunnel for the Linux boxes and they will forward all traffic for us.


Configuring the OpenSwan VPN tunnel.

Start off with a base Linux instance (14.04 max supported version at time of writing).

Run the following command to install OpenSwan.

apt-get install openswan


Edit /etc/ipsec.conf (sudo nano /etc/ipsec.conf) add following line

include /etc/ipsec.d/*.conf

Replace the following line





This will cause logging to end up in /tmp/pluto.log, which is exceptionally handy for debugging issues.



Edit /etc/ipsec.secrets

Comment out any previous includes (/var/lib/openswan/


add following line

include /etc/ipsec.d/*.secrets

The above basically redirects all ipsec settings to look within the ipsec.d folder, in which we will put our customisations below.

Private key exists in /etc/ipsec.d/private/ip-x-x-x-xKey.pem but we won’t use it here as authby=secret not rsakey


Create /etc/ipsec.d/vpc1-to-vpc2.conf

conn vpc1-to-vpc2

Create /etc/ipsec.d/vpc1-to-vpc2.secrets PSK "myprivatetopsecretcode"

(Same baseline configuration as above, up to the VPC1 line)

Create /etc/ipsec.d/vpc2-to-vpc1.conf

conn vpc2-to-vpc1

Create /etc/ipsec.d/vpc2-to-vpc1.secrets PSK "myprivatetopsecretcode"


Configure routing

The following settings will turn the Linux instance into a router, accepting all incoming packets and routing them via the OpenSwan VPN.
Edit /etc/sysctl.conf, add the following settings

net.ipv4.ip_forward = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0

Persist the settings

sysctl -p /etc/sysctl.conf

You can check a value has persisted post reboot with this command:-

cat /proc/sys/net/ipv4/ip_forward


Set up Firewall rules and NAT on both Linux VPCs

This must be configured on both Linux VPCs to forward to the other. Replace [VPNHostIP] with the corresponding side of the VPN.

iptables -t nat -A POSTROUTING -o eth0 ! -p esp -j SNAT --to-source [VPNHostIP]

iptables -A INPUT -p udp --dport 500 --j ACCEPT
iptables -A INPUT -p udp --dport 4500 --j ACCEPT
iptables -A INPUT -p esp -j ACCEPT


Restart IPSEC

sudo service ipsec restart

Checking VPN Status
The following commands can be helpful in checking or troubleshooting your VPN status:

sudo ipsec verify

(checks the status of the services required for OpenSWAN to run properly)

sudo service ipsec status

(checks the status of the OpenSWAN service and the VPN tunnels)


Automatically resizing Windows EC2, using Knife command -ebs-size

We’ve discovered an issue when trying to use the -ebs-size command within Chef. The instances are created, but due to a limitation with the AMIs they won’t use the full amount of disk space without a resize.

I’ve got a “Base” Windows recipe which I use across all Windows instances. I’ve applied the following Powershell code to resolve this problem for us :-

It checks to ensure that there is enough space to meet minimum requirements, then automatically extends the boot volume to the greatest value available.

powershell_script 'Extending primary volume' do
  code <<-EOH
	$drive = (Get-WmiObject Win32_OperatingSystem).SystemDrive.Replace(":", "")
	$minBytesToActivate = 1048576 #1MB minimum in Windows, you may wish to change this.

	$volume = Get-Volume -DriveLetter $drive
	$actualSize = $volume.Size
	$sizeMax = (Get-PartitionSupportedSize -DriveLetter $drive).SizeMax
	$diff = $sizeMax - $actualSize
	if ($diff -gt $minBytesToActivate)
		$diff, $minBytesToActivate
		Resize-Partition -DriveLetter $drive -Size $sizeMax

Now, if we choose to resize our EBS to 60GB, the drive is automatically resized for us.

Generic delimited parsing algorithm

Hi all,

You can use the following extension method to parse any string that is split by a delimiter. It will take care of most issues associated with splitting strings (quotes, etc) and works very well in almost every scenario I have used it in.

Good one for the library of extension methods, definitely.

        /// <summary>

        /// This method will attepmt to parse the string into its delimited parts, taking a delimiter string as a parameter.

        /// </summary>

        /// <param name="strLine">The string to parse.</param>

        /// <param name="fieldDelimiter">The string used as a delimiter in the parse string.</param>

        /// <returns></returns>

        public static IEnumerable<string> ParseDelimited(this string strLine, string fieldDelimiter)


            string separatorEscaped = Regex.Escape(fieldDelimiter);

            string regularExpression = @"^(?:""(?<item>[^""]*)""|(?<item>[^{0}]*))(?:{0}(?:""(?<item>[^""]*)""|(?<item>[^{0}]*)))*$";


            var regex = new Regex(string.Format(regularExpression, separatorEscaped));


            var split = regex





              .Select(c => c.Value)



            return split;


FTP – Get directory listing

Haven’t written anything on here for a while – so I am going to start getting back on it.

Here’s some nice code to get a directory listing from FTP. You get back a list of FTPListDetail objects which can be used to pass to further implementations, such as a download / downloadasync etc. It uses some nice regex to parse the FTP response.


        /// Returns a directory listing of the remote FTP host.



        public IEnumerable<FTPListDetail> GetDirectoryListing()


            var result = new StringBuilder();

            var request = GetWebRequest(WebRequestMethods.Ftp.ListDirectoryDetails);

            using (var response = request.GetResponse())


                using (var reader = new StreamReader(response.GetResponseStream()))


                    string line = reader.ReadLine();

                    while (line != null)




                        line = reader.ReadLine();


                    result.Remove(result.ToString().LastIndexOf(‘\n’), 1);

                    var results = result.ToString().Split(‘\n’);

                    string regex =

                        @”^” +               //# Start of line

                        @”(?<dir>[\-ld])” +          //# File size          

                        @”(?<permission>[\-rwx]{9})” +            //# Whitespace          \n

                        @”\s+” +            //# Whitespace          \n

                        @”(?<filecode>\d+)” +

                        @”\s+” +            //# Whitespace          \n

                        @”(?<owner>\w+)” +

                        @”\s+” +            //# Whitespace          \n

                        @”(?<group>\w+)” +

                        @”\s+” +            //# Whitespace          \n

                        @”(?<size>\d+)” +

                        @”\s+” +            //# Whitespace          \n

                        @”(?<month>\w{3})” +          //# Month (3 letters)   \n

                        @”\s+” +            //# Whitespace          \n

                        @”(?<day>\d{1,2})” +        //# Day (1 or 2 digits) \n

                        @”\s+” +            //# Whitespace          \n

                        @”(?<timeyear>[\d:]{4,5})” +     //# Time or year        \n

                        @”\s+” +            //# Whitespace          \n

                        @”(?<filename>(.*))” +            //# Filename            \n

                        @”$”;                //# End of line

                    foreach (var parsed in results)


                        var split = new Regex(regex)


                        var dir = split.Groups[“dir”].ToString();

                        var permission = split.Groups[“permission”].ToString();

                        var filecode = split.Groups[“filecode”].ToString();

                        var owner = split.Groups[“owner”].ToString();

                        var group = split.Groups[“group”].ToString();

                        var size = split.Groups[“size”].ToString();

                        var month = split.Groups[“month”].ToString();

                        var timeYear = split.Groups[“timeyear”].ToString();

                        var day = split.Groups[“day”].ToString();

                        var filename = split.Groups[“filename”].ToString();

                        yield return new FTPListDetail()


                            Dir = dir,

                            Filecode = filecode,

                            Group = group,

                            FullPath = CurrentRemoteDirectory + “/” + filename,

                            Name = filename,

                            Owner = owner,

                            Permission = permission,

                            Size = size.ToInt() ?? 0,

                            Month = month,

                            Day = day,

                            YearTime = timeYear







        /// Get the request using a specific URI


        ///<param name=”method”></param>

        ///<param name=”uri”></param>


        private FtpWebRequest GetWebRequest(string method, string uri)


            Uri serverUri = new Uri(uri);

            if (serverUri.Scheme != Uri.UriSchemeFtp)


                return null;


            var reqFTP = (FtpWebRequest)FtpWebRequest.Create(serverUri);

            reqFTP.Method = method;

            reqFTP.UseBinary = true;

            reqFTP.Credentials = new NetworkCredential(Connection.Username, Connection.Password);

            reqFTP.Proxy = null;

            reqFTP.KeepAlive = false;

            reqFTP.UsePassive = false;

            return reqFTP;


    public class FTPListDetail


        public bool IsDirectory




                return !string.IsNullOrWhiteSpace(Dir) && Dir.EqualsIgnoreCase(“D”);



        internal string Dir { get; set; }

        public string Permission { get; set; }

        public string Filecode { get; set; }

        public string Owner { get; set; }

        public string Group { get; set; }

        public int Size { get; set; }

        public string Name { get; set; }

        public string FullPath { get; set; }

        internal string Month { get; set; }

        internal string Day { get; set; }

        internal string YearTime { get; set; }

        public DateTime Date




                var month = DateTime.ParseExact(Month, “MMM”, CultureInfo.CurrentCulture).Month;

                if (!YearTime.Contains(“:”))


                    return new DateTime(YearTime.ToInt() ?? 0, month, Day.ToInt() ?? 0);




                    var dateTime = YearTime.Split(new string[] { “:” }, StringSplitOptions.RemoveEmptyEntries);

                    if (dateTime.Count() == 2)


                        int hour = dateTime[0].ToInt() ?? 0;

                        int minute = dateTime[1].ToInt() ?? 0;

                        return new DateTime(DateTime.Now.Year, month, Day.ToInt() ?? 0, hour, minute, 0);


                    return new DateTime(DateTime.Now.Year, Month.ToInt() ?? 0, Day.ToInt() ?? 0);




        public override string ToString()


            return string.Format(“{0} {1} {2} {3} {4} {5} {6} {7}”,












Refresh rate switching app

Hi all,

I was looking around for an application which sits in the system tray and watches for key combinations to automatically change your monitor refresh rate – I assumed somebody would have written something to do this, but couldn’t find a single app, so I wrote something.

I have uploaded it here :-

It creates an icon in the system tray – if you right click you will get a mapping of your current resolution to the different refresh rates supported .. if you change resolution it will update itself to the new available refresh rates. Just add it to your startup section or HKLM/Software/Microsoft/CurrentVersion/Run/ in the  registry to get it to auto-launch at startup.

To use it – simple note the key combinations .. they will be in the range of :-

Control+Alt+ F1 to Control+Alt+F10

If I get enough coffees bought for me – I might consider adding some sort of configuration section to it with user configurable key mappings and multi resolution support.

PS. It uses the Low Level Keyboard Hook which is mentioned in the following post :-

Copy paste from Excel into a DataTable

Ever wanted to copy and paste from Excel into a System.Data.DataTable for displaying in a DataGrid? Here’s some quick and dirty code to do just that :-

private void PasteFromExcel()

            DataTable tbl = new DataTable();
            tbl.TableName = "ImportedTable";
            List<string> data = new List<string>(ClipboardData.Split('\n'));
            bool firstRow = true;
            if (data.Count > 0 && string.IsNullOrWhiteSpace(data[data.Count - 1]))
                data.RemoveAt(data.Count - 1);
            foreach (string iterationRow in data)
                string row = iterationRow;
                if (row.EndsWith("\r"))
                    row = row.Substring(0, row.Length - "\r".Length);
                string[] rowData = row.Split(new char[] { '\r', '\x09' });
                DataRow newRow = tbl.NewRow();
                if (firstRow)
                    int colNumber = 0;
                    foreach (string value in rowData)
                        if (string.IsNullOrWhiteSpace(value))
                            tbl.Columns.Add(string.Format("[BLANK{0}]", colNumber));
                        else if (!tbl.Columns.Contains(value))
                            tbl.Columns.Add(string.Format("Column {0}", colNumber));
                    firstRow = false;
                    for (int i = 0; i < rowData.Length; i++)
                        if (i >= tbl.Columns.Count) break;
                        newRow[i] = rowData[i];
            this.WorkingTableElement.WorkingTable = tbl;
            tableImportGrid.DataSource = null;
            tableImportGrid.DataSource = tbl;

Custom WebBrowser Control with Zooming and CSS Injection

I thought I would post this custom browser I wrote for a project I am working on. It allows you to access the zoom function of the Internet Explorer browser (AxWebBrowser) by calling a Zoom method (passing in a whole number, expressed as percentage). I also added an InjectCSS() method which allows you to insert some extra CSS into the page (after it is loaded). I used this in a project recently where I didn’t want to mess with my generated page CSS or HTML, but simple wanted to add a couple of styles for display only purposes.

    public partial class CustomBrowser : WebBrowser
        [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        private static extern short GetAsyncKeyState(int keyCode);
        public bool IsKeyDown(Keys key)
            return (GetAsyncKeyState((int)key) & 0x8000) != 0;
        #region enums
        public enum OLECMDID
            // ...
            OLECMDID_OPTICAL_ZOOM = 63,
            // ...
        public enum OLECMDEXECOPT
            // ...
            // ...
        public enum OLECMDF
            // ...
            OLECMDF_SUPPORTED = 1
        #region IWebBrowser2
        [ComImport, /*SuppressUnmanagedCodeSecurity,*/
         TypeLibType(TypeLibTypeFlags.FOleAutomation |
                     TypeLibTypeFlags.FDual |
        public interface IWebBrowser2
            void GoBack();
            void GoForward();
            void GoHome();
            void GoSearch();
            void Navigate([In] string Url,
                          [In] ref object flags,
                          [In] ref object targetFrameName,
                          [In] ref object postData,
                          [In] ref object headers);
            void Refresh();
            void Refresh2([In] ref object level);
            void Stop();
            object Application
            object Parent
            object Container
            object Document
            bool TopLevelContainer { get; }
            string Type { get; }
            int Left { get; set; }
            int Top { get; set; }
            int Width { get; set; }
            int Height { get; set; }
            string LocationName { get; }
            string LocationURL { get; }
            bool Busy { get; }
            void Quit();
            void ClientToWindow(out int pcx, out int pcy);
            void PutProperty([In] string property,
                             [In] object vtValue);
            object GetProperty([In] string property);
            string Name { get; }
            int HWND { get; }
            string FullName { get; }
            string Path { get; }
            bool Visible { get; set; }
            bool StatusBar { get; set; }
            string StatusText { get; set; }
            int ToolBar { get; set; }
            bool MenuBar { get; set; }
            bool FullScreen { get; set; }
            void Navigate2([In] ref object URL,
                           [In] ref object flags,
                           [In] ref object targetFrameName,
                           [In] ref object postData,
                           [In] ref object headers);
            OLECMDF QueryStatusWB([In] OLECMDID cmdID);
            void ExecWB([In] OLECMDID cmdID,
                        [In] OLECMDEXECOPT cmdexecopt,
                        ref object pvaIn, IntPtr pvaOut);
            void ShowBrowserBar([In] ref object pvaClsid,
                                [In] ref object pvarShow,
                                [In] ref object pvarSize);
            WebBrowserReadyState ReadyState { get; }
            bool Offline { get; set; }
            bool Silent { get; set; }
            bool RegisterAsBrowser { get; set; }
            bool RegisterAsDropTarget { get; set; }
            bool TheaterMode { get; set; }
            bool AddressBar { get; set; }
            bool Resizable { get; set; }
        private IWebBrowser2 axIWebBrowser2;
        public CustomBrowser()
        protected override void AttachInterfaces(
            object nativeActiveXObject)
            this.axIWebBrowser2 = (IWebBrowser2)nativeActiveXObject;
        protected override void DetachInterfaces()
            this.axIWebBrowser2 = null;
        protected override void OnDocumentCompleted(WebBrowserDocumentCompletedEventArgs e)
        public void InjectCSS()
                mshtml.HTMLDocument test = (mshtml.HTMLDocument)this.Document.DomDocument;
                //inject CSS
                if (test.styleSheets.length < 31)
                { // createStyleSheet throws "Invalid Argument if >31 stylesheets on page
                    mshtml.IHTMLStyleSheet css = (mshtml.IHTMLStyleSheet)test.createStyleSheet("", 0);
                    css.cssText = "//Insert Custom CSS here!";
                    // CSS should now affect page
                    System.Console.WriteLine("Could not inject CSS due to styleSheets.length greater than 31");
            catch { }
        int currentZoom = 100;
        public void Zoom(int factor)
            object pvaIn = factor;
                   ref pvaIn, 
            catch (Exception)