This led to an interesting issue, we were using the webhook so we didn’t need to poll their API, now we need to poll their API on the off chance there is an error.

When looking for possible options to implement the polling I thought about using a SQS delay queue, delaying the message visibility by the polling interval. If the status isn’t completed or failed another message is queued again delaying the message visibility.

The general flow is below

The GeneratePDF posts a message to the queue containing the status_id that Doc Raptor give us to check the status, our export id and the original queue time with delay seconds set to 30 seconds. The poll lambda picks up the message and checks the sataus. If the status is completed the lambda just returns, if the status is failed it records the failed reason and returns. If the status is something else it posts another messag to the queue with same data and another 30 second delay. If we find that we have been polling for more than 15 minutes we raise an error and exit.

Lastly the code to send the delay message is very simple.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

import { SQS } from 'aws-sdk';

import config from '../config';
export interface pdfCheckMessage {
  exportId: string;
  statusId: string;
  startTime: string;
}

const POLL_QUEUE = config.get('pollQueue');
const DELAY_SECONDS = config.get('delaySeconds');

const sqs = new SQS();

export const queueDelayedMessage = async (message: pdfCheckMessage) => {
  await sqs
    .sendMessage({
      QueueUrl: POLL_QUEUE,
      MessageBody: JSON.stringify(message),
      DelaySeconds: DELAY_SECONDS,
    } as SQS.Types.SendMessageRequest)
    .promise();
};

Yes this code needs upgrading to use the AWS SDK V3.

1
2
3
4
5
6
7
8

{
  "message": "EMFILE: too many open files, open '/tmp/couchbackup/couchbackup-f..._2022-08-22T17:30:33.642Z.log'",
  "stack": "Error: EMFILE: too many open files, open '/tmp/couchbackup/couchbackup-f..._2022-08-22T17:30:33.642Z.log'",
  "errno": -24,
  "code": "EMFILE",
  "syscall": "open",
  "path": "/tmp/couchbackup/couchbackup-f..._2022-08-22T17:30:33.642Z.log"
}

Looking through our code we don’t directly open any files or sockets which would cause this type of issue.
We could see that file descriptors where being left open using Lambda Insights.

So, we Googled to see if anyone else had this issue and discovered this blog. It suggested adding a node package called wtfnode wtfnode - npm (npmjs.com). Unfortunately, this didn’t seem to work, the author of the package does say that this package is using undocumented Node apis and could break without warning.

After a few more weeks of Googling, I came across this package samswen/lambda-emfiles: solve node AWS lambda EMFILE issue (github.com). This package handles the EMFILE issue a different way, it causes the Lambda process to exit with a status code of 1. This means that lambda throws away the current lambda environment and starts a new one. While it didn’t fix the issue it meant that we didn’t get the retries when a process ran out of file descriptors.
I did a little poking around in the source code of this package to see how it was working out the number of open file descriptors. What I found was

1
2
3
4
5
6
7
8
9
10
11
12

return new Promise((resolve) => {
  exec(`ls /proc/${process.pid}/fd`, (err, stdout, stderr) => {
    if (err || stderr) {
      this.is_ok = false;
      resolve(this.is_ok);
    } else {
      const parts = stdout.split("\n");
      this.emfiles_count = parts.length - 1;
      resolve(this.is_ok);
    }
  });
});

The thing that supprised me was that the Lambda run time has access to the /proc virtual directory. I guess that makes sense as it is running on Linux.

Having found this I had a play around on my machine to see what information you could get. I found that ls -l returned

1
2
3
4
5
6
7
8
9
10
11

lr-x------ 1 sbx_user1051 990 64 Nov 29 17:46 0 -> /dev/null
l-wx------ 1 sbx_user1051 990 64 Nov 29 17:46 1 -> pipe:[3890]
lrwx------ 1 sbx_user1051 990 64 Nov 29 17:46 10 -> anon_inode:[eventpoll]
lrwx------ 1 sbx_user1051 990 64 Nov 29 17:47 100 -> /tmp/tmp-33-k3p5q3R0D4Fi
lrwx------ 1 sbx_user1051 990 64 Nov 29 17:47 101 -> /tmp/tmp-33-7uD5r8I2FL2J
lrwx------ 1 sbx_user1051 990 64 Nov 29 17:47 102 -> /tmp/tmp-33-x5ZwfUpaOOSp
lrwx------ 1 sbx_user1051 990 64 Nov 29 17:47 103 -> socket:[13171]
lrwx------ 1 sbx_user1051 990 64 Nov 29 17:47 104 -> /tmp/tmp-33-JNd7W44OhPj8
lrwx------ 1 sbx_user1051 990 64 Nov 29 17:47 106 -> socket:[13173]
lrwx------ 1 sbx_user1051 990 64 Nov 29 17:47 108 -> /tmp/tmp-33-I7NKSkv4kw8P
lrwx------ 1 sbx_user1051 990 64 Nov 29 17:47 109 -> /tmp/tmp-33-qymMaVIo1Zga

After finding this we added some more logging to the lambda to record this before the process finished and what we found was the number of tmp files kept increasing. We now had out smoking gun.

Now that we knew what we were looking for I had a look to check that our code wan’t creating temp files, and we weren’t.

Then I started having a look at the Cloudant couchbackup library that we use. The first thing I noticed was in their package.json they referenced a package tmp, so it looks like they use tmp files. Next, I looked for references to that package and found

1
2
3
4
5
6
7
8
9
10

function apiDefaults() {
  return {
    parallelism: 5,
    bufferSize: 500,
    requestTimeout: 120000,
    log: tmp.fileSync().name,
    resume: false,
    mode: "full",
  };
}

Ok, so firstly does this get called when we use their backup function and secondaly what does it do? More digging and I found it is aliased app.js and it is called in the backup function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

backup: function(srcUrl, targetStream, opts, callback) {
    const listenerErrorIndicator = { errored: false };
    if (typeof callback === 'undefined' && typeof opts === 'function') {
      callback = opts;
      opts = {};
    }
    if (!validateArgs(srcUrl, opts, callback)) {
      // bad args, bail
      return;
    }

    // if there is an error writing to the stream, call the completion
    // callback with the error set
    addEventListener(listenerErrorIndicator, targetStream, 'error', function(err) {
      debug('Error ' + JSON.stringify(err));
      if (callback) callback(err);
    });

    opts = Object.assign({}, defaults(), opts);

Right, so it is called in the path we are using what does it do? Next, I dig into the tmp package.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

/**
 * Synchronous version of file.
 *
 * @param {Options} options
 * @returns {FileSyncObject} object consists of name, fd and removeCallback
 * @throws {Error} if cannot create a file
 */
function fileSync(options) {
  const args = _parseArguments(options),
    opts = args[0];

  const discardOrDetachDescriptor =
    opts.discardDescriptor || opts.detachDescriptor;
  const name = tmpNameSync(opts);
  var fd = fs.openSync(name, CREATE_FLAGS, opts.mode || FILE_MODE);
  /* istanbul ignore else */
  if (opts.discardDescriptor) {
    fs.closeSync(fd);
    fd = undefined;
  }

  return {
    name: name,
    fd: fd,
    removeCallback: _prepareTmpFileRemoveCallback(
      name,
      discardOrDetachDescriptor ? -1 : fd,
      opts,
      true
    ),
  };
}

The first thing I noticed was on line 15 it is opening a file and then on like 18 is closes the file only if you pass in opts.discardDescriptor === true. That looks like the issue!!

Lastly before raising an issue with Cloudant I wanted to test it, so in our couch-backup lambda I went into the node_modules folder, found the Cloudant code and changed tmp.fileSync().name to tmp.fileSync({discardDescriptor : true}).name and push those changes to production.

The next day this was the graph from Lambda Insights

Yay!!! We have found the problem.

The last thing was raising an issue with Cloudant File descriptor leak · Issue #535 · cloudant/couchbackup (github.com) and they are implementing what I suggested fix file descriptor leak in backup by emlaver · Pull Request #536 · cloudant/couchbackup (github.com)

For a litte while I was trying to figure out how to stop CDK from deleting old lambda version so you could roll back quickly. Turns out it is very simple, you just need to set the rentention policy of the version to retain.

1
2
3
4
5
6
7
8
9
10
11

var function = new Function(this, "example-lambda", new FunctionProps
{
  <other set up code>
  CurrentVersionOptions = new VersionOptions
  {
      RemovalPolicy = RemovalPolicy.RETAIN
  }
}

var version = function.CurrentVersion;
version.AddAlias("production");

Which gives this in the lamba console

Full code here

I originally had the class defined as

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

public class TagAspect :  IAspect
{
    private readonly IList<string> _requiredTags;

    public TagAspect(IList<string> requiredTags)
    {
        _requiredTags = requiredTags;
    }

    public void Visit(IConstruct node)
    {
        if (node is Stack stack)
        {
            foreach (var requiredTag in _requiredTags)
            {
                if (!stack.Tags.TagValues().ContainsKey(requiredTag))
                {
                    Annotations.Of(node).AddError($"Missing required tag {requiredTag} on stack with id {stack.StackName}");
                }
            }
        }
    }
}

But when I ran CDK synth I got this error

1
2

Unhandled exception. System.ArgumentException: Could not convert argument 'DeployLambda.TagAspect' to Jsii (Parameter 'arguments')
   at Amazon.JSII.Runtime.Deputy.DeputyBase.<>c__DisplayClass20_0.<ConvertArguments>b__0(Parameter parameter, Object frameworkArgument)

After a bit of Googling I found there is a known issue with an easy fix. You just need to make sure the aspect class inherites from Amazon.Jsii.Runtime.Deputy.DeputyBase

The working code is

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

public class TagAspect : Amazon.JSII.Runtime.Deputy.DeputyBase, IAspect
{
    private readonly IList<string> _requiredTags;

    public TagAspect(IList<string> requiredTags)
    {
        _requiredTags = requiredTags;
    }

    public void Visit(IConstruct node)
    {
        if (node is Stack stack)
        {
            foreach (var requiredTag in _requiredTags)
            {
                if (!stack.Tags.TagValues().ContainsKey(requiredTag))
                {
                    Annotations.Of(node).AddError($"Missing required tag {requiredTag} on stack with id {stack.StackName}");
                }
            }
        }
    }
}

You wire this in to the program.cs

1

Aspects.Of(app).Add(new TagAspect(new List<string>{"owner"}));

Which gives this error is we don’t have a owner tag set

1
2

[Error at /lambda-pipeline] Missing required tag owner on stack with id lambda-pipeline
Found errors

Excample code

There was a new feature added to CDK in 2020 called bundling which seems to now be the correct way for deploying lambdas via CDK. With bunding CDK downloads an AWS container, map the source folder to that container, I then use dotnet publish which builds the code in the container. The output from the the .Net publish command is put in the containers /asset-output folder. This is used by the pipeline to get the compiled code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

IEnumerable<string> commands = new[]
{
    "cd /asset-input",
    "dotnet publish -c Release -r linux-arm64 -p:PublishReadyToRun=true -o /asset-output"
};

var function = new Function(this, "example-lambda", new FunctionProps
{
    FunctionName = "ToUpper-Lambda",
    Description = "Changes input to upper case",
    Runtime = Runtime.DOTNET_CORE_3_1,
    Handler = "ExampleLambda::ExampleLambda.Function::FunctionHandler",
    Architecture = Architecture.ARM_64,
    Code = Code.FromAsset("./ExampleLambda/", new AssetOptions
    {
        Bundling = new BundlingOptions
        {
            Image = Runtime.DOTNET_CORE_3_1.BundlingImage,
            Command = new []
            {
                "bash", "-c", string.Join(" && ", commands)
            }
        }
    })
});

Once you have the lambda stack set up, create a stage for the lambda deployment. When this is added to the pipeline you will notice that an Assets stage is added to the pipeline.

The other thing you need to do is make sure the step that synth stack is run in a CodeBuildStep, not a ShellStep. It also needs to be privileged as this is the step that will build the lambda code inside a container as well synth the stack.

When you run the pipeline you will see the lambda is built in the build stage, then the lambda is taken from the build assets and moved into it’s one asset/s3 key so that it can be passed to the cloudformation stack that cdk creates.

I’ve put the full example on Github

While Codespaces will correctly start a container with the the required node tools it doesn’t have the AWS cli which is needed for deployments from the Serverless framework. It tuns out the fix is quite simple and the Github documenation covers a lot of what you need.

The main part of it is you need to create a .devcontainer folder in your repo. Inside this folder you need a Docker file and a devcontainer.json file (Notice there is no . in the file name only in the folder name). Github has a repo with the base conatiners for the language you are using.

I used the Node container and created this Dockerfile.

1
2
3
4
5
6

ARG VARIANT=16
FROM mcr.microsoft.com/vscode/devcontainers/javascript-node:${VARIANT}

RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \
        unzip awscliv2.zip && \
        sudo ./aws/install

The devcontainer.json just contains

1
2
3
4
5
6
7
8
9

{
    "name": "Node.js",
"build": {
"dockerfile": "Dockerfile",
// Update 'VARIANT' to pick a Node version: 12, 14, 16
"args": { "VARIANT": "16" }
}
}

Which is just copied from the GitHub template.

The next bit was I didn’t really want to store my credenitals in the container and GitHub and thought of that. I created to variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with my credentials from AWS.

The last thing was rebuilding the CodeSpace and Restarting it, both of wich you do in VSCode from the command pallet.

After that try the aws cli.

Checking the logs, we had started getting this error.

1
2
3
4
5
6
7
8
9
10

System.IO.IOException: The configured user limit (128) on the number of inotify instances has been reached, or the per-process limit on the number of open file descriptors has been reached.
   at System.IO.FileSystemWatcher.StartRaisingEvents()
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.TryEnableFileSystemWatcher()
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.CreateFileChangeToken(String filter)
   at Microsoft.Extensions.Primitives.ChangeToken.OnChange(Func`1 changeTokenProducer, Action changeTokenConsumer)
   at Microsoft.Extensions.Configuration.FileConfigurationProvider..ctor(FileConfigurationSource source)
   at Microsoft.Extensions.Configuration.Json.JsonConfigurationSource.Build(IConfigurationBuilder builder)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at Microsoft.Extensions.Configuration.SystemsManagerExtensions.GetAwsOptions(IConfigurationBuilder builder)
   at Microsoft.Extensions.Configuration.SystemsManagerExtensions.AddSystemsManager(IConfigurationBuilder builder, Action`1 configureSource)

The other thing that had changed was that due to having to encrypt the clusters EBS volumes we had used this as an opportunity to upgrade the version of Linux the cluster was using.

After a little bit of Googling this appeared to be due to the reloadOnChange flag being set to true when adding our Json config files. Since we are running this in a container and containers are immutable it seemed to be an easy fix, just set reloadOnChange to false.

When we deployed the updated version, we ended up with the same exception, from the call stack it appeared that .Net was ignoring the value of the flag. Very confusing 😕.

I ended up debugging into the Microsoft code to see what was going on and found that we were using Host.CreateDefaultBuilder adds the config files itself with reload on change set to true as you can see in the code snippet below.

1
2
3
4
5
6
7
8

builder.ConfigureAppConfiguration((hostingContext, config) =>
{
    IHostEnvironment env = hostingContext.HostingEnvironment;
 
    bool reloadOnChange = hostingContext.Configuration.GetValue("hostBuilder:reloadConfigOnChange", defaultValue: true);
 
    config.AddJsonFile("appsettings.json", optional: true, reloadOnChange: reloadOnChange)
            .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: reloadOnChange);

Link to this code on MS GitHub

Microsoft was setting up the Json config files as well as our code, and theirs was defaulting to reloadOnChange. This call is pointless in our solution as we are then doing all this set up ourselves. If you read the rest of the CreateDefaultBuilder file there is stuff in there we don’t use and stuff we end up replacing, so why are we using it?

The fix was easy, change Host.CreateDefaultBuilder to new HostBuilder. Digging into the history of Host.CreateDefaultBuilder (thanks MS for putting it on GitHub), it turns out this function has worked the same way since at least .Net core 2.0. In .Net 5 they have added a flag so you can set reloadOnChange to false, but as all our applications do this type of setup themselves it is pointless to us.

The learnings from this are to check what sample code does, given that all the source for .Net core/.Net 5 is one GitHub check what the MS calls are doing. From what found it appears that we have been doing our set up wrong for a while as we end up duplicating a lot of what is in Host.CreateDefaultBuilder. Going forward I’ll be using new HostBuilder, unless I’m doing a simple app that only needs what is in Host.CreateDefaultBuilder.

Anyway, I passed and am certified for another 3 years.

From the Microsoft blog

“OpenTelemetry is created as a merger of the OpenCensus and OpenTracing projects. This project aspires to make robust, portable telemetry a built-in feature of any software. It allows you to collect low-overhead telemetry from the entirety of a modern software stack.”

I originally saw mention of OpenTelemetry on the ASP.Net core blog and then this month Jimmy Bogard started twitting about a project he was playing with, some of which he has blogged and appears to be an on going series. This got me interested in having another look at it.

What does it offer us

In short OpenTelemery allows us a consistent way of adding end-to-end diagnostics and tracing to an application, extracting away the system we are using to visualise this information. Currently there is a Alpha library for .Net https://github.com/open-telemetry/opentelemetry-dotnet which should be moving to beta shortly.

I don’t have a full example, I’ll leave that up to Jimmy as he will do a much better job, but I have some highlights below.

To configure OprnTelemetry for NewRelic we would configure it like this

1
2
3
4
5
6
7
8

services.AddOpenTelemetry((sp, builder) =>
    {
        builder
          //.SetSampler(Samplers.AlwaysSample)
          .UseNewRelic(this.Configuration)
          .AddRequestCollector()
          .AddDependencyCollector();
     });

NewRelic have their own library for OpenTelemetry.

AWS are starting to look at OpenTelemetry for XRay as well, but only Java so far :sad:.

Wrap up

I feel this is something that is worth keeping an eye on. It seems to have good support from vendors and is supported by the Cloud Native Computing Foundation.

If we had this when we switched from DataDog the migration would have been as simple as changing .UseDataDog to .UseNewRelic.

It turned out to be really easy to set up. All I needed to do was place a pre-push file into the projects .git/hooks folder.

1

dotnet test ./src/project.sln

So now when a do a push from the shell

I also use the git client built in to Visual Stuido. This also seems to respect the hooks although it doesn’t give feed back on progress.

Since this site is hosted in S3 it was pretty much a matter of following the AWS documentation. The only issue I had was configuring Route53 where the documentation is a little lacking.

In Route53 when you set up the Alias record you set the target to the DNS name of your CloudFront distribution. The appears to only be need is you already have your S3 bucket set up for static site hosting.

Windows 10 has had a bash shell for a while now, which I have been using for SSH and which works very well. What some people may not be aware of is that Microsoft added a full native OpenSSH client to the Windows 10 2018 spring update.

To use it you need to enable the Windows feature. You do this bu going into settings and searching for features and selecting “Manage Optional Features”.

Click the + button at the top and find OpenSSH Client in the list and click install. That’s it you now have a ssh client that you can run from Powershell or CMD.

The last thing you might need to do is if your private key is password protected and you want to add it to the ssh-agent.

1
2
3
4
5
6
7
8
9
10
11
12
13

Get-Service ssh-agent

Status   Name               DisplayName
------   ----               -----------
Stopped  ssh-agent          OpenSSH Authentication Agent

Start-Service ssh-agent

Status   Name               DisplayName
------   ----               -----------
Running  ssh-agent          OpenSSH Authentication Agent

ssh-add id_rsa

All done, you can now use OpenSSH for you Git client as well as logging into Linux machines.

The last part of my Alexa blog as been posted.

Thanks for the Echo I just have to figure out how to program it now.

In .Net 4.6 Dapper uses DataTables and the .AsTableValuedParameter helper method. DataTables don’t exist in .Net core which means that we have an issue.

What I found is that you can go back to what appears to be the original way that Dapper supported TVPs and create a class that implements IDaynamicParameters.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

public class EmailAddressesParameter : SqlMapper.IDynamicParameters
  {
      private readonly IEnumerable<string> _emailAddresses;

      /// <summary>Initializes a new instance of the
      /// <see cref="T:System.Object" /> class.</summary>
      public EmailAddressesParameter(IEnumerable<string> emailAddresses)
      {
          _emailAddresses = emailAddresses;
      }

      /// <summary>
      /// Add all the parameters needed to the command just before it executes
      /// </summary>
      /// <param name="command">The raw command prior to execution</param>
      /// <param name="identity">Information about the query</param>
      public void AddParameters(IDbCommand command, SqlMapper.Identity identity)
      {
          var sqlCommand = (SqlCommand) command;
          sqlCommand.CommandType = CommandType.StoredProcedure;
          var items = new List<SqlDataRecord>();

          foreach (var emailAddress in _emailAddresses)
          {
              var rec = new SqlDataRecord(new SqlMetaData("EmailAddress",
                SqlDbType.NVarChar, 100));
              rec.SetString(0, emailAddress);
              items.Add(rec);
          }

          var p = sqlCommand.Parameters.Add("@emailAddress", SqlDbType.Structured);
          p.Direction = ParameterDirection.Input;
          p.TypeName = "EmailTableType";
          p.Value = items;
      }
  }

Then you call it like this.

1
2
3
4
5

using (var connection = new SqlConnection(connectionString))
{
  var parameters = new EmailAddressesParameter(new List<string>() { "test@test.com" });
  connection.Execute("test", parameters);
}

It has a slight downside that you need to create a new class for each TVP you use. I don’t think that is too big of a deal as it would be a one time set up.

For some reason these tests stated failing on from our build server where they where working locally. The issue was where we had a post back from an external site, when the request hit our site it was getting a 404 error.

So I started looking at what was different between our local machines and what the build server was using. On our local machines we run a Selenium hub locally, but the build server users a server which turned out to be running Linux.

Having found that I set up a Linux VM so I could see Chrome running. So watching the test what I saw was this

This was very odd as the route did exist and it worked fine when running on Windows.

Digging into the code one thing I did notice was that we weren’t defining the scheme on the URL when constructing the call back URL. I added the scheme to the url and it started working.

So it appears while Chrome on Windows was able to infer the scheme, Chrome on Linux needed it explicitly defined.

One way to do this is

1
2
3
4
5

public IActionResult DoSomething()
{
  var headerValues = request.Headers.GetValues("MyHeader");
  var header = headerValues.FirstOrDefault();
}

YUCK.

I wasn’t very keen on this and starting think it must be possible to use WebApi’s model binding. After a bit of searching I found https://blog.bennymichielsen.be/2013/11/17/modelbinding-with-headers-in-asp-net-webapi/ which gave me most of
the details. This left me with model binding working, but not quite in the way I wanted.

1
2
3
4

public IActionResult DoSomething([ValueProvider(typeof(HeaderValue))] int custId)
{
  //Do something with custId
}

Better, but not quite what I wanted.

In trying to fill in a few of the missing pieces I had a look at the WebApi code on GitHub and found that in one of their Integration tests Microsoft had pretty much exactly what I was after.

So between the two I was able to put together something I was happy with.

1
2
3
4
5
6
7
8
9
10
11
12
13

public class HeaderValueProviderFactory : ValueProviderFactory
{
    /// <summary>
    ///     Gets the value provider.
    /// </summary>
    /// <param name="actionContext">The action context.</param>
    /// <returns></returns>
    public override IValueProvider GetValueProvider(HttpActionContext actionContext)
    {
        return new HeaderValueProvider(actionContext.Request);
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

/// <summary>
/// Returns the tenant id from the request header.
/// http://blog.bennymichielsen.be/2013/11/17/modelbinding-with-headers-in-asp-net-webapi/
/// also some ideas from the ASP.NET unit tests
/// https://github.com/ASP-NET-MVC/aspnetwebstack/blob/4e40cdef9c8a8226685f95ef03b746bc8322aa92/test/System.Web.Http.Integration.Test/ModelBinding/ModelBindingController.cs#L271
/// </summary>
public class HeaderValueProvider : IValueProvider
{
    private readonly HttpRequestMessage _requestMessage;

    /// <summary>
    /// Initializes a new instance of the <see cref="HeaderValueProvider"/> class.
    /// </summary>
    /// <param name="requestMessage">The request message.</param>
    public HeaderValueProvider(HttpRequestMessage requestMessage)
    {
        _requestMessage = requestMessage;
    }

    /// <summary>
    /// Determines whether the specified prefix contains prefix.
    /// </summary>
    /// <param name="prefix">The prefix.</param>
    /// <returns></returns>
    public bool ContainsPrefix(string prefix)
    {
        return _requestMessage.Headers.Contains(prefix);
    }

    /// <summary>
    /// Gets the value.
    /// </summary>
    /// <param name="key">The key.</param>
    /// <returns></returns>
    public ValueProviderResult GetValue(string key)
    {
        IEnumerable<string> headerValue;
        ValueProviderResult result = null;
        if (_requestMessage.Headers.TryGetValues(key, out headerValue))
        {
            var header = headerValue as IList<string> ?? headerValue.ToList();
            if (header.Any())
            {
                var value = header.First();
                result = new ValueProviderResult(value, value, CultureInfo.InvariantCulture);
            }
        }
        return result;

    }

}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method | AttributeTargets.Parameter, Inherited = true, AllowMultiple = false)]
public class FromHeaderAttribute : ModelBinderAttribute
{
    /// <summary>
    /// Gets the value providers that will be fed to the model binder.
    /// </summary>
    /// <returns>
    /// A collection of <see cref="T:System.Web.Http.ValueProviders.ValueProviderFactory"/> instances.
    /// </returns>
    /// <param name="configuration">The <see cref="T:System.Web.Http.HttpConfiguration"/> configuration object.</param>
    public override IEnumerable<ValueProviderFactory> GetValueProviderFactories(HttpConfiguration configuration)
    {
        return base.GetValueProviderFactories(configuration).OfType<HeaderValueProviderFactory>();
    }
}

1
2
3
4
5
6
7
8

 public static class WebApiConfig
 {
     public static void Register(HttpConfiguration config)
     {
         // Web API configuration and services
         config.Services.Add(typeof(ValueProviderFactory), new HeaderValueProviderFactory());
     }
}

So finally I can just do

1
2
3
4

public IActionResult DoSomething([FromHeader(Name="CustId")] int custId)
{
  // Do something with custId
}

Much nicer.

So I’ve moved it to Hexo because I wanted a static site generator that wasn’t written in Ruby and would run on Windows. Hexo is pretty good, but I’ve some some of the plug ins need Python 2 and then need to try an compile C code which works about as bladly as it does for Ruby.

I event tried it under the new Windows 10 bash support, but I seems that node isn’t node, but is nodejs. I need to do some more research on that.

The idea of user secrets is that you need to have some settings on you dev machine which you don’t want in source control.  Things like passwords etc.

I’ve found the documentation some what lacking so hopefully this overview will help.

Firstly you need to install the user-secret command. The installation instructions aren’t too bad and can be found here DNX-Secret-Configuration.

To add the user secrets

Once I had that installed I added the userSecrestsId to my project.json, but that didn’t seem to be picked up.

{      "webroot": "wwwroot",      "userSecretsId": "aspnet5-4c570577-d4ba-491c-a56c-3f267e3d5211",      "version": "1.0.0-*",

I found that I need to add this to the startup.js when I loaded the user secrets.

public class Startup      {          public Startup(IHostingEnvironment env)          {              var builder = new ConfigurationBuilder();              if (env.IsDevelopment())              {                  builder.AddUserSecrets("aspnet5-4c570577-d4ba-491c-a56c-3f267e3d5211");              }              else              {                  builder.AddEnvironmentVariables();              }              Configuration = builder.Build();          }

The next problem was figuring out how to inject these values into the controller.  What I found is you need to AddOptions in the ConfigureServices method and they Configure the class you want the values put in.

public void ConfigureServices(IServiceCollection services)         {             services.AddMvc();             services.AddOptions();             services.ConfigureAwsOptions>(Configuration);

My AwsOptions class just has two string fields with the name of the secret properties I want to use.

namespace Config  {      public class AwsOptions      {          public string awsAccessKey { get; set; }          public string awsSecret { get; set; }              }  }

The ASP.NET configuration system will look for values in the Configuration object that match these name and load them into that class.  Then to get them in the controller you do the following

[Route("api/[controller]")]  public class AController : Controller  {      private readonly IOptionsAwsOptions> _awsOptions;      private readonly ILoggerSeverController> _logger;      public AController(IOptionsAwsOptions> awsOptions, ILoggerAController> logger)      {          _awsOptions = awsOptions;          _logger = logger;      }      // POST api/values      [HttpPost]      public async TaskIActionResult> Post(Guid Id)      {          AmazonSimpleNotificationServiceClient snsclient =  new AmazonSimpleNotificationServiceClient(_awsOptions.Options.awsAccessKey,                   _awsOptions.Options.awsSecret, RegionEndpoint.USWest2);

That’s it.  Then you will get your user secret values injected into you class.

Full Startup.cs (because I hate it when the snippets miss something I need)

using Microsoft.AspNet.Builder;  using Microsoft.AspNet.Hosting;  using Microsoft.Framework.Configuration;  using Microsoft.Framework.DependencyInjection;  using Microsoft.Framework.Logging;  using Severing.Service.Config;  namespace Service  {      public class Startup      {          public Startup(IHostingEnvironment env)          {              var builder = new ConfigurationBuilder();              if (env.IsDevelopment())              {                  builder.AddUserSecrets("aspnet5-4c570577-d4ba-491c-a56c-3f267e3d5211");              }              else              {                  builder.AddEnvironmentVariables();              }              Configuration = builder.Build();          }          public IConfiguration Configuration { get; set; }          // This method gets called by a runtime.          // Use this method to add services to the container          public void ConfigureServices(IServiceCollection services)          {              services.AddMvc();              services.AddOptions();              services.ConfigureAwsOptions>(Configuration);              services.AddLogging();              // Uncomment the following line to add Web API services which makes it easier to port Web API 2 controllers.              // You will also need to add the Microsoft.AspNet.Mvc.WebApiCompatShim package to the 'dependencies' section of project.json.              // services.AddWebApiConventions();          }          // Configure is called after ConfigureServices is called.          public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)          {              loggerFactory.AddConsole();              // Configure the HTTP request pipeline.              app.UseStaticFiles();              // Add MVC to the request pipeline.              app.UseMvc();              // Add the following route for porting Web API 2 controllers.              // routes.MapWebApiRoute("DefaultApi", "api/{controller}/{id?}");          }      }  }

What I’ve ended up having to do is this.

1
2
3

MessageConventionExtensions.IsMessageTypeAction =
  t => t.Namespace != null && t.Namespace.EndsWith(“Messages”) && !t.Namespace.StartsWith(“NServiceBus”);
Test.Initialize();