The Ops Community ⚙️: Simon Foster

AWS Cloud Practitioner

Simon Foster — Thu, 17 Mar 2022 20:00:45 +0000

So today I sat the AWS Cloud Practitioner Certification Exam and passed! This is my second certification exam I have sat after the Azure Fundamentals one I sat last year.

This time was different than the Azure one. Colleagues at work encouraged me to look into doing this particular exam and my employer is going to cover the cost of the exam itself. Last time I did the exam in my own time and didn’t really discuss it with anyone from work.

The other difference was this time I went to a testing centre. The centre was a 5 minute walk from the office. So I decided to work from the office today and 20 minutes before the exam started I walked to the centre. I signed a couple of bits of paper, had my photo taken, had my passport and driving license laughed at, put my belongings in a locker and I was free to start the exam. No worrying about internet connectivity or kids making noise outside my room, or the endless photos at the prep stage of an exam from home. I think I prefer the exam centre experience.

The other difference was the first exam was Azure, this one was AWS. Both exams cover similar concepts, they are both introductions to Cloud Computing. It’s ten months since I did the Azure one, however they both had multiple choice questions and gave you the results at the end. I feel overly familir with Azure, while I don’t feel that was with AWS yet. A lot of questions relied on you knowing what AWS services are called. eg AWS Pipeline, AWS CodeStar, AWS CodeBuild, Amazon CodeGuru are all AWS services and are easily mixed up, especially as I have only used a few AWS services, while I have used a lot more of the Azure ones.

My memory is the Azure gave you a breakdown of your score at the end, while I am still waiting for that from todays exam. This difference may of course be due to it being in a testing centre.

I still prefer Azure to AWS, however today has proven that I can learn AWS stuff as well. For my next exam I want to get more hands on with my learning, as most of the services I have learnt about so far I haven’t seen in action.

Updating SQL Server with Docker

Simon Foster — Wed, 23 Feb 2022 00:00:00 +0000

This morning I was listening to a podcast where the new features coming out for SQL Server 2022 were being discussed. This starting me thinking about what would be involved in upgrading.

Upgrading production environments is complex and there are licensing considerations to take into account. However for non production workloads like development this isn’t a problem so lets look at that first.

In the past I have installed SQL Server Devloper Edition onto my laptop, this is fine but I have found that unless you are very careful you may end up with multiple different versions of SQL Server sitting around, and it is difficult to cleanly remove them without a fresh install of the OS.

However in this day and age, Docker and Containers are king. My current development environment makes use of Docker and has a docker compose file which sets up SQL Server for this particular application, lets take a look.

  sqlserver:
    image: mcr.microsoft.com/mssql/server:2019-latest
    container_name: Sql
    ports:
      - "5432:1433"
    networks:
      - my-network
    volumes:
      - sqlvolume:/var/opt/mssql

This defines which docker image to use, in this case 2019-latest, sets up ports and the name and saves the data on a docker volume.

If we then run SELECT @@VERSION on this instance of SQL Server we get told:

Microsoft SQL Server 2019 (RTM-CU13) (KB5005679) - 15.0.4178.1 (X64) Sep 23 2021 16:47:49 Copyright (C) 2019 Microsoft Corporation Developer Edition (64-bit) on Linux (Ubuntu 20.04.3 LTS) <X64>

What if we change the docker-compose file to use 2022-latest?

manifest for mcr.microsoft.com/mssql/server:2022-latest not found: manifest unknown: manifest tagged by "2022-latest" is not found

SQL Server 2022 hasn’t been released yet so there is no docker image for it yet. Try this command again in a few months when it is available.

OK so what else can we try? What about a downgrade to 2017-latest? Will that work?

SQL Server 2017 starts but the following error gets logged.

2022-02-23 21:41:15.30 Server Software Usage Metrics is disabled.

2022-02-23 21:41:15.30 spid6s Starting up database 'master'.

2022-02-23 21:41:15.34 spid6s Error: 948, Severity: 20, State: 1.

2022-02-23 21:41:15.34 spid6s The database 'master' cannot be opened because it is version 904. This server supports version 869 and earlier. A downgrade path is not supported.

Doh we can’t downgrade the existing database we have. Probably a good thing really.

Microsoft release regular updates for SQL Server called CU’s (Cumulative Updates), you can see above we are on CU13. Is there a CU14 or CU15 we could try?

Update the docker compose to: mcr.microsoft.com/mssql/server:2019-CU14-ubuntu-20.04

At this point I actually got an error

2022-02-23 21:50:20.71 Server Error: 17058, Severity: 16, State: 1.

2022-02-23 21:50:20.71 Server initerrlog: Could not open error log file '/var/opt/mssql/log/errorlog'. Operating system error = 5(Access is denied.).

This is caused by trying to use SQL Server 2017 but it is easy to fix.

In docker desktop there is a volumes section, find the volume you are using with SQL Server and delete the errorlog mentioned above.

Now if you retry SQL will start OK.

Repeating the SELECT @@VERSION gives us a new CU

Microsoft SQL Server 2019 (RTM-CU14) (KB5007182) - 15.0.4188.2 (X64) Nov 3 2021 19:19:51 Copyright (C) 2019 Microsoft Corporation Developer Edition (64-bit) on Linux (Ubuntu 20.04.3 LTS) <X64>


Microsoft SQL Server 2019 (RTM-CU15) (KB5008996) - 15.0.4198.2 (X64) Jan 12 2022 22:30:08 Copyright (C) 2019 Microsoft Corporation Developer Edition (64-bit) on Linux (Ubuntu 20.04.3 LTS) <X64>

How much easier is this than manually installing updates and rebooting or attempting to uninstall and reinstall SQL Server. As SQL Server 2022 isn’t out yet I can’t say for certain what issues I may encounter but hopefully it will be as easier as this. And I don’t need to backup or restore and databases they are all available as before!

Using GitHub Actions

Simon Foster — Mon, 10 Jan 2022 20:00:45 +0000

I’ve been running my website on Azure Static Web Apps for a while and it is pretty cool.

When you create a Static Web App on Azure you get asked for the github repo of your source code and even the branch to use.

Once you have selected this, you get asked for the type of code to deploy, mine is Blazor Web Assembly but you can use Angular, React or Vue.

You now have three variables to fill in the location in your code of the Website, the location of your Azure Functions and the output location usually wwwroot. Once you have set these three you can preview the GitHub Actions file that will be created and added to your repository.

I get something like this

name: Azure Static Web Apps CI/CD

on:
  push:
    branches:
      - feature/tempbranch
  pull_request:
    types: [opened, synchronize, reopened, closed]
    branches:
      - feature/tempbranch

jobs:
  build_and_deploy_job:
    if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.action != 'closed')
    runs-on: ubuntu-latest
    name: Build and Deploy Job
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: true
      - name: Build And Deploy
        id: builddeploy
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_<GENERATED_HOSTNAME> }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
          action: "upload"
          ###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
          # For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
          app_location: "Client" # App source code path
          api_location: "Api" # Api source code path - optional
          output_location: "wwwroot" # Built app content directory - optional
          ###### End of Repository/Build Configurations ######

  close_pull_request_job:
    if: github.event_name == 'pull_request' && github.event.action == 'closed'
    runs-on: ubuntu-latest
    name: Close Pull Request Job
    steps:
      - name: Close Pull Request
        id: closepullrequest
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_<GENERATED_HOSTNAME> }}
          action: "close"

This github action will run when you create a Pull Request to the branch mentioned in the file, or if you push code into the branch. This code get added into the .github/workflows/ folder and is the location that all github action workflows live.

I haven’t done much with github actions, however I have used Azure DevOps quite a bit. Over on the Azure DevOps side I have created a pipeline that deploys to a Dev environment, then a Test environment and finally a production environment.

Lets have a look at the workflow that I ended up with and with can break down how it all works. Note I am new to Github actions so if there is a better way of doing this do let me know.

name: Azure Static Web Apps

on:
  push:
    branches:
      - main
      - develop
      - feature/*

jobs:
  dev:
    runs-on: ubuntu-latest
    environment: 
      name: Dev
    name: Dev
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: true
      - name: Build And Deploy
        id: builddeploy
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_ORANGE_POND_09B18B903 }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
          action: "upload"
          ###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
          # For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
          app_location: "Blog" # App source code path
          api_location: "Blog.Func" # Api source code path - optional
          output_location: "wwwroot" # Built app content directory - optional
          ###### End of Repository/Build Configurations ######
  test:
    if: github.ref == 'refs/heads/develop'
    runs-on: ubuntu-latest
    environment: 
      name: Test
    name: Test
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: true
      - name: Build And Deploy
        id: builddeploy
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_WITTY_DUNE_0A1A77903 }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
          action: "upload"
          ###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
          # For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
          app_location: "Blog" # App source code path
          api_location: "Blog.Func" # Api source code path - optional
          output_location: "wwwroot" # Built app content directory - optional
          ###### End of Repository/Build Configurations ######
  prod:
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: 
      name: Prod
    name: Prod
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: true
      - name: Build And Deploy
        id: builddeploy
        uses: Azure/static-web-apps-deploy@v1
        with:
          azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_BRAVE_ROCK_0AAC63D03 }}
          repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for Github integrations (i.e. PR comments)
          action: "upload"
          ###### Repository/Build Configurations - These values can be configured to match your app requirements. ######
          # For more information regarding Static Web App workflow configurations, please visit: https://aka.ms/swaworkflowconfig
          app_location: "Blog" # App source code path
          api_location: "Blog.Func" # Api source code path - optional
          output_location: "wwwroot" # Built app content directory - optional
          ###### End of Repository/Build Configurations ######

The first thing I did was create three Azure Static Web Apps, I am using the free tier so while this is trippling my costs it is all still free! Doing this created three github action workflow files, I deleted two and edited the third, but before I deleted them I made a note of the AZURE_STATIC_WEB_APPS_API_TOKEN. If you look in your settings -> secrets for your repo you will see secrets have been created, this is the secure token that github uses to update your static web app.

While we are in settings we might as well look at environments. I created a Prod, Test and Dev environment that I was going to use in my github actions.

Environments can have various rules setup on them.

Required reviewers - this is like an approver, a user specified here must aprove for the workflow to be deployed
Wait time - I didn’t use this, but it looks like a certain amount of time can be set to pause the deployment. (I assume to do some kind of manual check)
Deployment Branch - specify what branch are allowed to be deployed to what environments. I specified develop, main and feature branches could be deployed to the Dev environment, develop and main could go on Test and main could go on Prod
Environment secrets - I didn’t use this as my secrets were already created, however it looks like your secrets can be associated with a specific environment

Now that we have the static web apps setup and the environments lets look at the github action file.

First of all I removed the PR stuff and just concentrated on pushes. I wanted my workflow to be.

Push to feature branch
Deploys to Dev env
PR feature branch to develop
Once merged code gets pushed into develop
Deploys to Test env
PR develop to main
Once merged code gets pushed into main
Deploys to Prod env (after approval)

The approval on deploying to production I think is probably overkill, but I still have it setup like that for now.

My gh action has three jobs defined as dev: test: and prod: they are all the same except they have the azure_static_web_apps_api_token that is correct for their environment.

They also each have a environment defined eg

environment:
  name: Prod

Lastly Test and Prod have an if test setup, if the test is false the job won’t run. Importantly it won’t fail it just won’t run.

For Prod this needs to only run on main branch so we have

if: github.ref == ‘refs/heads/main’

For Test this needs to only run on develop so

if: github.ref == ‘refs/heads/develop’

I could have a test for develop to only run on feature/* but I have allowed it to run everytime.

There is loads more you can do with github actions, but hopefully this gives you a taste of some of the things you can do. I currently have a mix of Azure DevOps and github actions so I will be working on getting github actions to do more.

Azure DevOps Release Pipelines Pre and Post Approval

Simon Foster — Sun, 14 Feb 2021 20:00:45 +0000

Azure DevOps release pipelines have lots of options to do things how you want. One of my favourites is the option for approval.

There are two ways you can do approvals Pre and Post deployment. Lets look at both.

Pre Deployment Approval

Lets imagine you have a simple deployment pipeline that deploys to a test/development environment before deploying to a production environment.

Pre Deployment Approval happens immediately before the release so in this example, click in the ellipse before the Prod release step.

You will get a screen like the above, you can select what users need to approve it and how long approval waits before timing out, the default is 30 days, but I tend to use a shorter time out of 3 days.

Post Deployment Approval

Post Deployment Approval happens immediately after the release so in this example, click in the circle after the Test release step.

You will get a screen like the above, with the same settings as before.

That is pretty much all there is to approvals so either option will prompt you to approve before anything gets deployed to your production environment.

Deployment Hours

To complicate matters I make use of the following setting to define deployment hours.

This setting will start the Prod deployment at 3am Mon-Fri.

If I configure Post Deployment Approval, as soon as my deploy to Test has completed a request for Approval is sent.

If I configure Pre Deployment Approval, at 3am Mon-Fri a request for Approval is sent (not ideal if you tend to be asleep at 3am)

So it looks like Post Deployment Approval is more useful for my use case. However if you deny approval either in Pre or Post approval this will mark the deployment as failed and show Red in your list of deployments.

From a casual glance it looks like the deployment to Test is failing, it isn’t I am just opting to not continue my deployment to production.

My Pipeline

This is how I have my pipeline setup. Deployment happens on Test and doesn’t have a post approval step.

After Test an empty stage called Approval runs and that has a post deployment approval, this happens immediately after Test so you get asked straight away for approval.

Prod does not start as I have my deployment hours configured. Once it is time for deployment to Prod to start it executes.

Now a casual look at my past releases, you can easily see which have been stopped by approval and which have failed due to whatever issue, and which have run all the way through to Prod.

And deployments to Prod can only ever run during my defined deployment window.

I am interested to hear how you have your deployment pipeline setup. Do you make use of Pre or Post Approvals? Do you ensure deployments always happen at specific times?

Testing for expiring SSL Certificates

Simon Foster — Tue, 03 Mar 2020 20:00:45 +0000

Let’s Encrypt is amazing, you can easily add SSL certificates to any website and automate the renewal process. I have talked before about how impressive it is.

Once you start adding SSL certificates to your production sites however you may want to check when they expire so you don’t get caught out. You can always open your site in your favourite browser and view the certificate information and expiry date.

However there is a way to automate this check.

[Fact]
public void IsSSLExpiring()
{               
  var handler = new HttpClientHandler
  {
    ServerCertificateCustomValidationCallback = CustomCallback
  };
  var client = new HttpClient(handler);

  HttpResponseMessage response = client.GetAsync("https://www.example.com").GetAwaiter().GetResult();
  Assert.True(response.IsSuccessStatusCode);
}

private bool CustomCallback(HttpRequestMessage arg1, X509Certificate2 arg2, X509Chain arg3, SslPolicyErrors arg4)
{
  var now = DateTime.UtcNow;
  var expire = arg2.NotAfter;
  var diff = (expire - now).TotalDays;

  Assert.InRange(diff, 30, 1000);
  return arg4 == SslPolicyErrors.None;
}

This code gets the SSL expiry date from https://www.example.com and will fail the xunit test if the expiry date is less than 30 days in the future. I then schedule my tests to run regularly on all my environments with a Let’s Encrypt Certificate and this gives me advanced warning if a SSL certificate is about to expire.

The Assert.InRange(diff, 30, 1000) line will fail the test if the expiry date is less than 30 days or greater than 1000, but as the default expiry for Let’s Encrypt certificates is three months it will never be greater than 1000 days even with a freshly installed certificate. These values can be tweaked to suit your use case, however 30 days is enough time for me to investigate what is happening.

To execute my tests I use a scheduled build in Azure DevOps, but anything that regularly can run your tests will do the job.

The code above is just a simple example to get your started for my purposes I have put all my URLs into config files and just pass these into my tests, so I don’t need a custom test for every different URL.

Gated Release

Simon Foster — Fri, 05 Apr 2019 20:00:45 +0000

Automated releases of software are great but how can we add an element of feedback so only good releases go live.

I have been using Azure DevOps to release my PwnedPass android app to the Google Play Store for a while now. There are options to deploy to the alpha, Beta or Production tracks and even to set % of users to target. For the full range of options check out the Google Play extension for Azure DevOps.

My release starts by publishing to 10% of users on the production track, my next step makes use of the increase rollout option to increase this %, you can have as many of these additional steps as you want until you reach 100% of your users.

Now if you run this release now it will just run through each of the steps one after the other. Now of course you can add a pre or post approval to your pipeline but this just adds a manual dependency to your release. Whoever does the approving needs to check things are working before approving or worse just approves regardless.

Azure DevOps has the concept of gated releases which allows you to add automated checks before or after a release happens. These automated checks can be any of the following:

An Azure Function
A Rest API call
Azure Monitor Alert
Query Work Items
Security and Compliance Assessment

We are going to make use of the Azure Monitor Alert, to create an alert from your Application Insights data and only continue the rollout if no failures are detected.

Open up your application insights resource in the Azure portal and look in alerts. Click add new alert rule.

Select your application insights resource in Resource, In Condition choose a condition to check, I chose Failed Requests, so every time a failure is registered in my API I can stop the deployment. The exact criteria you want to use is entirely up to you.

Create an action group, I just set my alert to send an email to myself but there are other alert actions you may want to try. Give your alert a name and description and click save.

Now all we need to do is make Azure DevOps make use of this alert. In your release pipeline select the pre-deployment conditions of your second step and open up the Gates section.

Choose a suitable time to evaluate, I have been using something long like 12 or 24 hours so if there are problems there is time for it to be noticed. Choose Version 1 of the task (I was not able to get it to work with Version 0)

Now select your Azure subscription and Resource Group and leave the rest of the settings as they are. Now your Deployment will stop and analyse application insights for any Failed requests and will halt if it finds any.

I am still testing this out but it will take a few days to figure out if this what I want due to the large time scales involved. I feel this is going to be an improvement of manually approving release steps.

Azure Key Vault

Simon Foster — Tue, 19 Mar 2019 20:00:45 +0000

Azure Key Vault is a secure way of storing your keys, certificates and secrets so your application can access everything it needs to but you don’t have them being stored insecurely anywhere such as in source control.

I have been wanting to give Azure Key Vault a try for a while now as it can make use of Azure Active Directory to give your web app an identity so it can authenticate itself into the key vault to access secrets. Pretty clever but with a lot of moving parts a bit complex.

For my example I am just going to connect to my Key Vault and get a secret and display it somewhere on a web page. This is of course not what you want to do as secrets are secret and shouldn’t be displayed just used to authenticate into whatever, however it is an easy way to prove I am connecting to the Key Vault and everything is working.

Lets look at some code. I have a .net core application and to start with lets install three nuget packages.

Microsoft.Azure.KeyVault
Microsoft.Azure.Services.AppAuthentication
Microsoft.Extensions.Configuration.AzureKeyVault

I’ve not include version numbers as these will no doubt get updated over time but hopefully it will still work.

Now in your Program.cs add the following code, replacing [KeyVaultName] with the name of your Key Vault.

    public class Program
    {
        public static void Main(string[] args)
        {
            CreateWebHostBuilder(args).Build().Run();
        }

        public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
            WebHost.CreateDefaultBuilder(args)
                .ConfigureAppConfiguration((context, config) =>
            {
                    var builtConfig = config.Build();

                    var azureServiceTokenProvider = new AzureServiceTokenProvider();
                    var keyVaultClient = new KeyVaultClient(
                        new KeyVaultClient.AuthenticationCallback(
                            azureServiceTokenProvider.KeyVaultTokenCallback));

                    config.AddAzureKeyVault(
                        $"https://[KeyVaultName].vault.azure.net/",
                        keyVaultClient,
                        new DefaultKeyVaultSecretManager());
            })
            .UseApplicationInsights()
            .UseStartup<Startup>();
    }

Now all you need to do is look at your configuration to pull out secrets from your Azure Key Vault. If you have a secret called AppSecret then you can use the following code snippet to retrieve its value, assuming _configuration is an implementation of Microsoft.Extensions.Configuration.IConfiguration.

_configuration["AppSecret"];

Now if you do all of this and run from an Azure Web App or run locally it will fail to pull anything from the Key Vault. You need to give your web app an identity and configure your key vault to allow access from that identity.

Once my code has been deployed to an Azure Web App I get the following error.

Lets look at fixing that, first lets give my web app an Identity. Open up the Azure portal and find the identity section of your web app and turn the setting on.

Now you need to grant that identity permission to your key vault. In the portal open up Access Policies in your key vault and click add Policy, select the identity of your web app in the principal box and select the following settings to grant access to your secret.

Now you have a website that can pull secrets out of Key Vault but only that unique identity. Anyone who has access to your source code will not have access to your secrets, even if they push your code to a different Azure Web App.

Yaml Builds on Azure DevOps

Simon Foster — Thu, 31 Jan 2019 20:00:45 +0000

I have been using Azure DevOps (Or VSTS or VSO etc) for a while now and one of the great features is doing automatic builds with every check-in. This is more commonly known as a CI (continuous integration) build.

More recently I have started playing about with creating my build using YAML files instead of using the web user interface to create my build.

Why?

You may wonder why go to the effort of learning the YAML syntax when you can just create the build in Azure DevOps and then forget about it.

Mostly it is because the build changes over time and you shouldn’t just forget about it. If something changes over time then you might want to version control it, or look at a previous version.

Lets say you create a pull request that replaces a .net 4.7 web service with a .net core web service. If you have a CI build this PR will fail because it won’t build. If you change the build first any other builds going on will fail. What you want in this case is the build to be associated with that branch or PR. Any builds before you merge this change in will continue to work and any after this change will also work.

How?

How do you get started with YAML builds? Well the first thing is to make sure that YAML builds are turned on, as I write this I believe they are still a feature you can turn on or off. Have a look in Preview features and make sure they are turned on.

Next look at any of your existing builds and click the View YAML link. This will show you an example YAML file of your existing build. You could just save this as azure-pipelines.yml and checkin to the root of your project. You can also click the add new build pipeline option, this will give you some templates to start you off.

The YAML file consists of a series of build steps usually called tasks, with a few settings before to configure things like parameters or build agents. Detailed docs about the syntax of the file can be found here.

For my mobile app my YAML files consist of downloading nuget packages, building the solution, building specific projects with desired settings, running powershell or other scripts to set things up and finally publishing the results of the build as artifacts so that they can be used in any releases.

Secure It!

Avoid committing passwords and secure keys into source control. I have found you can upload secure files via Azure DevOps and then add a download secure files step at the start of your build. This allows the secure file to be used during the build but the contents of the file can’t be viewed by anyone with access to the source code.

I find it often takes a bit of thinking about how to achieve this, but it is usually possible to keep keys and secrets secure.

Running SQL Server on a Linux Container using Docker for Windows

Simon Foster — Mon, 05 Nov 2018 00:00:00 +0000

Recently I have been investigating what all the fuss is about Docker and it has been well worth my time as Docker is pretty awesome for automating stuff.

My development environment has typically required installing SQL Server. SQL is a bit of a beast with lots of options and takes time to setup how you want.

However since Microsoft have now created a version of SQL Server that runs on Linux you can run SQL Server in a Linux container with only a few commands.

I am going to assume you already have Docker for windows installed on your development machine. If not head over to Docker and find out how.

The Microsoft guide to setting up SQL Server in a Linux container can be found here.

First you need to download the image. In a powershell window run:

docker pull mcr.microsoft.com/mssql/server:2017-latest

This downloads the latest sql server image.

To run this image run the following:

docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=password"
-p 1433:1433 --name sql
-d mcr.microsoft.com/mssql/server:2017-latest

To run a SQL Server image you are required to accept the terms and conditions and set a default sa password. These are added as environment variables with the -e flag.

You also need to set the ports that your container will run on (1433 is the default SQL port) and give your container a name, in this case “sql”.

If you have already installed SQL Server you will not be able to run the container on the same port as your local install. To solve this you can select a different port.

docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=password" 
-p 1434:1433 --name sql
-d mcr.microsoft.com/mssql/server:2017-latest

-p 1434:1433 maps the 1433 port on the container to port 1434 of your local environment.

Once you have run this command you can connect SQL Server Management Studio (SSMS) to (local) or (local),1434 if you are using a different port using the credentials you provided and execute any SQL you like.

If your development environment requires windows authentication this of course is not for you, if it doesn’t you are good to go.

The development environment I have been using has various powershell scripts for setting things up. These assume windows auth. However I have adapted them to take custom credentials.

$credential = Get-Credential $server.ConnectionContext.LoginSecure=$false 
$server.ConnectionContext.set_Login($credential.UserName) 
$server.ConnectionContext.set_SecurePassword($credential.Password)

The Get-Credential command creates a dialog where you can enter SQL credentials, this is then stored in a variable and used in the rest of the script.

How do I restore a backup file to my container?

Run:

docker exec -it sql mkdir /var/opt/mssql/backup
docker cp database.bak sql:/var/opt/mssql/backup

This creates a backup folder and copies a backup file from your local environment to the container. You can then use management studio to restore the backup file (or you could write a sql script to do it). One thing to note when restoring databases, make sure the files are restored to Linux locations not windows locations.

The only issues I have encountered so far are the lack of support for SSIS packages and no windows auth. There are sql server windows images available which I haven’t tried yet which may work better with some of these options.

Let’s Encrypt is awesome

Simon Foster — Mon, 30 Apr 2018 00:00:00 +0000

Let’s Encrypt is a free way to get a SSL certificate onto your website and until recently I had never tried it. It is very easy and I think it is awesome.

IIS is the web server software the Microsoft include with Windows 10 and Windows Server. I have it installed on my laptop and it displays the default IIS page.

It is not really a good idea to host websites on your laptop, use a dedicated web server, or host with a hosting company, however the techniques are the same and it gives me something to write about!

In order to point a domain name at what IIS on my machine was serving up I did the following:

Do a google search for “whats my IP”, this will return your public IP. Most residential ISPs use dynamic IPs so it may change over time, (which is another reason not to host a website on your laptop!)
Add an A record on a domain with the IP address you have just got
Your public IP most likely points at your router not your laptop so enable port forwarding of port 80 and port 443 to the internal IP of your laptop (something like 192.168.0.11 etc)

Now comes the fun Let’s Encrypt stuff!

First you need a Let’s Encrypt client, there are a lot of them out there mostly for linux flavours, however a bit of googling found a windows one. Go to https://github.com/PKISharp/win-acme/releases and download the zip file and unzip it.

Run the executable from the zip file and follow the onscreen prompts.

Press N to create a new certificate.

Then press 1 to bind to single website found in your IIS setup

And now magically Let’s Encrypt knows what you have setup in IIS.

Now all you need to do is enter an email address incase a renewal fails and agree to the let’s encrypt terms and you are all setup.

How awesome and easy is that for getting your websites working with a SSL certificate. If you have IIS configured on a server, give it a try and you can SSL all your things.

GitHub history

Simon Foster — Mon, 24 Apr 2017 20:00:45 +0000

I created my GitHub account in August 2010, lets look at what I have done with it.

In 2010 I started by committing the code for an old php website I had created. Then in December I committed some other php sites.

Between 12 December 2010 and 29 September 2011 I must have created some automated process which is why there is a commit every day. Looking at the diff it appears to be related to tracking visitors to a site. I assume back then I hadn’t heard of the .gitignore file!

Nothing for a few years until 2014, when I started compiling a code samples collection, this is bits of code that I want to show off, I have added to this since 2015 so I really should go back to this as my skills have developed a bit since then.

In 2015 I really started to think of myself as a developer, I added repositories for Raspberry Pi, and a few C# ideas I had. I ended the year taking part in Advent of Code

What did I do last year? Well I started going to York Code Dojo and this meant lots of forks from their code examples and also my first pull request. I also did a fair bit of looking at other repositories trying to find a good open source project to contribute to, still not found one for me yet. Lastly I ended 2016 with a bit of Advent of Code.

Not sure how enthralling a blog post this is, but fascinating to see the different commit histories.

Automatic Git Tagging

Simon Foster — Thu, 16 Jun 2016 20:00:45 +0000

One of the features of git is the ability to tag a point in my change history with a tag. For a while now I have been manually tagging my code whenever I do a release, so I can easily work out what has changed by doing a diff between two tags.

Now that I am automating my release process with TeamCity I am thinking about how to manage my tags better.

TeamCity has a setting called VCS Labeling which comes in very handy.

Configuring it is fairly simple as it only has three settings.

VCS root to label : This is obviously the url to your git repository Labeling pattern : This is the text of the label to be added. Label successful builds only : Do you really want to add a tag if the build failed?

A tag needs to have a unique name, so adding a tag just called deployed won’t work. When I used to add tags manually I used the naming convention of deployedyyyymmdd. While this naming convention is possible with TeamCity I use something a bit more complex to provide more information about what has been deployed.

TeamCity provides lots of parameters that can be used in your build steps and also in the Labeling pattern box. I started off using deployed-build-%system.build.number% as my tag which just marks git with the TeamCity build number.

When I run a TeamCity deployment I don’t always use the same configuration options, I deploy locally, to a test server or to production and sometimes I just deploy the frontend or the backend. How cool would it be to include this information in the tag text?

Well the next step was to change my Labeling pattern to deployed-build-%system.build.number%-%ServerName%-%DatabaseName%-%FrontEndPath% , this adds the backend database config settings and the path the frontend was deployed to. Now when looking at git you can see commits marked with multiple tags, one for each deployment that succeeded and the tag will indicate the settings used during that deployment.

Now I will never forget to add the tag after a release as the adding of a tag is part of the deployment process, if the deployment fails the tag won’t be added. I can test my deployment to test and git will show if this has been successful, and when I deploy live this will also show up.

How do you use tags? Do you mark successful builds with a tag? Why not let me know or leave a comment below.