• ## Building Caffe on Windows using Visual Studio 201…3

Note that now that BVLC’s caffe repository directly supports compilation on Windows, this guide has become obsolete.

When attempting to check out Caffe after some wasted hours of getting TensorFlow to run on Windows (which sort-of works using Bash for Windows), I gave Caffe a try. However, the last time I tried to get a huge system of nested dependencies using CMake to work on Windows, things only got worse with every additional project. For Caffe, you’d need Boost, OpenCV, HDF and protobuf as well as some Google logging and command-line argument things amongst other stuff – and boy, make sure you got your linking, threading and runtime right, and don’t even think about mixing different C++ or Boost flavors.

So there’s this Windows fork of Caffe managed by Microsoft which apparently only requires you to configure the CommonSettings.props file to your liking and then compile. The nice part is, all references are pulled using NuGet. The bad part is, the packages are all MSVC 1800, i.e. Visual Studio 2013. You’ll be able to find newer variants of Boost, but that’s about it.

If you’re really trying to use Visual Studio 2015, fix the nuget.config to have the repository path fixed like described in this StackOverflow answer:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
</packageSources>
</configuration>


Also note that the path is most likely outside your build tree … but that’s how it is provided by the maintainer.
I wasted another day trying to get Caffe to work in Visual Studio 2015 by selecting the 2013 toolkit but, in the end, it came down to the following error:

NuGet error: unknown command 'overlay'


This overlay stuff apparently is some NuGet 2 magic that didn’t survive the dark ages, so it won’t work with NuGet 3, which I have in my path and which is also used internally by VS 2015.
There are a couple of questions regarding that error, like this one on the OpenCV Answers site, issues on GitHub (with half-broken answers since the CoApp tooling doesn’t really work exist anymore), etc., but there’s no obvious solution – at least if you’re using a modern Windows development environment – that is, one that’s using NuGet 3 instead of NuGet 2.6.

You can fix most of the problems by just upgrading OpenCV 2 to OpenCV 3. For that, kick out the old NuGet packages and reference the opencvdefault one as shown in this video. But even if you do that, it’ll fail on the glog 0.3.3 NuGet package which still uses overlays.

As it turns out though, the NuGet package for glog 0.3.3 already contains a NuGet binary at glog.0.3.3.0/build/native/private/nuget.exe, but since it’s not in the path, it won’t use it. Copy that to the package’s root (where NuGet-Overlay.cmd can be found and you’re set.

For the rest, make sure you’re using Python 2.7 though, as the Boost libraries are requiring that; also install NumPy.
Using e.g. Anaconda Python, you’d want

conda create -n caffe python=2.7 numpy


Which will leave you with the task to find python27_d.lib … or just build Release.

• ## Rust: adding arbitrary things

I just started learning Rust some days ago and, coming from a C/C++/C# background, immediately stubbed my toe on the type system. The notion of traits, however, is something I really like and allows for a stunt that is impossible in C# and ugly in C++: adding things, where by things I mean all the things.

The key is to constrain a generic type parameter with the Add<T> trait. In this example, I’m constraining a generic type T1 to be Add<T2>, that is, to be able to add a T2 to itself. The result of such an operation is obviously generic as well, <T1 as Add<T2>>::Output (with Output being the type), allowing for the following code:

fn add<T1, T2>(a:T1, b:T2) -> <T1 as Add<T2>>::Output
{
a+b
}


While this function does nothing more than adding two values of arbitrary types (try that in C# with primitive types), the underlying mechanism allows for interesting operations such as vector-matrix multiplication without having to resort to a concrete type. A Complex would work as well as a primitive f32 (i.e. float), f64 (i.e. double), etc.
You can do this in C++, but instead of having the compiler check the traits (like rustc does), you rather have to hope that someone implemented that specific operator overload or else you’d be in template error message hell.

• ## “Compiler crashed with code 1” on Mono

Update:
I opened a bug ticket for the described issue and it got fixed within the week.

Original post:
At work we recently stumbled across an interesting problem: While compilation of our C# 6 code base would work fine on Windows, a guy that works on OSX using Xamarin Studio suddenly was unable to compile the code, because “Error: Compiler crashed with code 1”. No line number, no nothing, only that.

After a run through git bisect we eventually were able to track down the commit where I found the error to lie within an await happening in a string interpolation. Sadly, this problem was introduced by simply (and automatically) refactoring all occurrences of String.Format() in the file.

I was able to reproduce the problem with the following simple code:

private async Task<int> Foo()
{
return 42;
}

{
Console.WriteLine($"Something {await Foo()}"); }  Apparently, the mono compiler doesn’t like that too much. The solution to this is simple: Don’t await within the interpolated string. Instead, do it outside: private async Task Bar() { var foo = await Foo(); Console.WriteLine($"Something {foo}");
}


And that’s that.

• ## Fun with ASP.NET 5 beta7: Bliss, Build errors, Dockerfiles and Reverse Proxies

I was never a ASP.NET developer, but since self-hosting on Linux only meant trouble, I gave ASP.NET vNext a try. At first, the whole DNX and .NET Core thing hit me like a hammer. A big one. Wielding a jackhammer. After working with it for a couple of days now though, I’m actually more than excited – it’s almost heresy. With all the logging, configuration services (Microsoft.Framework.Configuration!), developer secrets, host-integrated dependency injection included by design – not to forget the Linux support – there’s absolutely no way I’ll ever go back to .NET once DNX is stable.

That said, here are some things I stumbled over:

• If you just installed Visual Studio 2015 Community and ASP.NET is still beta, make sure to install the newest beta integration. Last time I checked I was two versions behind and things change rapidly at the moment.
• When diving into this DNX (“Class Library (Package)”, “Console Application (Package)”, ASP.NET 5 web project) thing, if you don’t explicitly need .NET Core support, kick the dnxcore50 framework out of the projects.json. That reduces pain time to a minimum, when otherwise you’d be searching for the cause of one gazillion build errors that should not be there.
• When using the official (and sweet) microsoft/aspnet Dockerfile, know that the examples are tailored for single-project solutions. Below is how I did it in the end.
• At least as per beta 7, the dnu restore in the docker build process takes ages. This has likely something to do with mono and – as per this bug report – there’s a quick fix for it: Add ENV MONO_THREADS_PER_CPU 2000 to your Dockerfile and see the world with new eyes.
• I also had to write a middleware for MVC 6 to support X-Forwarded-Proto and the official RFC 7239 Forwarded headers, more below.

First of all, I had my development site running locally in Visual Studio, available to the subnet through a local nginx reverse proxy which I would use to dispatch to any of my development websites. In this case, the .../aspnet route would point to my self-hosted application. That nginx, in turn, was behind another nginx, listening to the https://myhost/testing route on the actual webserver. Sadly, this base path of /testing/aspnet was totally tripping up ASP.NET which assumed a base path of /, also the protocol was not recognized. So I added this configuration to my local nginx:

upstream local_aspnet {
server localhost:9000;
}

server {
listen       80;
server_name  myhost localhost;

location /aspnet {
rewrite /testing/aspnet(.*) $1 break; # Handled within ASP.NET proxy_set_header X-Base-Path /testing/aspnet; proxy_set_header X-Real-IP$remote_addr;
proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto "https"; proxy_pass http://local_aspnet/; proxy_redirect off; proxy_set_header HOST$host;
proxy_buffering off;
break;
}
}


And implemented the following in C#:

public static void UseReverseProxyProtocolRecognition(
[NotNull] this IApplicationBuilder app)
{
app.Use(next => context =>
{
var request = context.Request;

{
}

Uri.UriSchemeHttps,
StringComparison.OrdinalIgnoreCase))
{
request.Scheme = Uri.UriSchemeHttps;
}
{
// http://tools.ietf.org/html/rfc7239
@"proto=(?<proto>https?)");
if (match.Success)
{
request.Scheme = match.Groups["proto"].Value;
}
}
return next(context);
});
}


which allowed me to simply do:

public void Configure(IApplicationBuilder app, IApplicationEnvironment env,
ILoggerFactory loggerfactory)
{
// ...
app.UseReverseProxyProtocolRecognition();
// ...
}


As for the Dockerfile, since I was using multiple projects in one solution, I had to cheat a little. This file works by copying the project files into the image, then running dnu restore individually for each DNX project, then adding the whole sources. This – when putting the least modified project first – allows for caching as much as possible when rebuilding the image, while still allowing for cross-project dependencies. Also note the use of MONO_THREADS_PER_CPU which really is a life-saver here.

FROM microsoft/aspnet

ENV STARTUP_PROJECT src/Startup.Project

COPY NuGet.Config /app/
COPY global.json /app/

COPY src/Startup.Project/project.json /app/src/Startup.Project/
COPY src/Second.Project/project.json  /app/src/Second.Project.Models/
COPY src/Third.Project/project.json   /app/src/Third.Project/

WORKDIR /app/src/Startup.Project
RUN ["dnu", "restore"]

WORKDIR /app/src/Second.Project
RUN ["dnu", "restore"]

WORKDIR app/src/Third.Project
RUN ["dnu", "restore"]

COPY . /app
WORKDIR /app/\$STARTUP_PROJECT

EXPOSE 5004
ENTRYPOINT ["dnx", "kestrel"]


If you’re at it, check out the Docker for Windows toolkit which amazingly sucks less than it did some months ago.
Also have fun and coffee. You know I do.

• ## Quadratic interpolation given two points and one derivative

While reading up on line search algorithms in nonlinear optimization for neural network training, I came across this problem: Given a function $f(x)$, find a quadratic interpolant $q(x) = ax^2 + bx + c$ that fulfills the conditions $f(x_0) = q(x_0)$, $f(x_1) = q(x_1)$ and $f'(x_0) = q'(x_0)$. Basically this:

So I took out my scribbling pad, wrote down some equations and then, after two pages of nonsense, decided it really wasn’t worth the hassle. It turns out that the simple system

\begin{align} f(x_0) &= ax_0^2 + bx_0 + c\\ f(x_1) &= ax_1^2 + bx_1 + c\\ f'(x_0) &= 2ax_0 + b \end{align}

for

\begin{align} q(x) &= ax^2 + bx + c \end{align}

has the solution

\begin{align} a &= – \frac{f(x_0) – f(x_1) – x_0 f'(x_0) + x_1 f'(x_0)}{(x_0 – x_1)^2} \\ b &= – \frac{x_0^2 f'(x_0) – x_1^2 f'(x_0) – 2x_0 f(x_0) + 2x_0 f(x_1)}{(x_0 – x_1)^2} \\ c &= \frac{x_0^2 f(x_1) + x_1^2 f(x_0) – 2x_0 x_1 f(x_0) – x_0 x_1^2 f'(x_0) + x_0^2 x_1 f'(x_0)}{(x_0 – x_1)^2} \end{align}

Instead of ruining your time on the paper, it can be obtained more easily in Matlab using

syms a b c x_0 x_1 f(x_0) f(x_1) df(x_0)
[a, b, c] = solve(...
f(x_0) == a*x_0^2 + b*x_0 + c, ...
f(x_1) == a*x_1^2 + b*x_1 + c, ...
df(x_0) == 2*a*x_0 + b, ...
a, b, c);

syms q(x)
q(x) = simplify(a*x^2 + b*x + c);


Obviously, the whole purpose of this operation is to find an approximation to the local minimizer of $f'(x)$. This gives

\begin{align} 0 &\overset{!}{=} q'(x_{min}) \\ x_{min} &= -\frac{1}{2} \frac{x_0^2 f'(x_0) -x_1^2 f'(x_0) – 2 x_0 f(x_0) + 2 x_0 f(x_1)} {f(x_0) – f(x_1) – x_0 f'(x_0) + x_1 f'(x_0)} \end{align}

We also would need to check the interpolant’s second derivative $q”(x_{min})$ to ensure the approximated minimizer is indeed a minimum of $q(x)$ by requiring $q”(x_{min}) > 0$, with the second derivative given as:

\begin{align} q”(x) &= – 2 \frac{f(x_0) – f(x_1) – x_0 f'(x_0) + x_1 f'(x_0)}{\left( x_0 – x_1 \right)^2 } \end{align}

The premise of the line search in minimization problems usually is that the search direction is already a direction of descent. By having $0 > f'(x_0)$ and $f'(x_1) > 0$ (as would typically be the case when bracketing the local minimizer of $f(x)$), the interpolant should always be (strictly) convex. If these conditions do not hold, there might be no solution at all: one obviously won’t be able to find a quadratic interpolant given the initial conditions for a function that is linear to machine precision. In that case, watch out for divisions by zero.

Last but not least, if the objective is to minimize $\varphi(\alpha) = f(\vec{x}_k + \alpha \vec{d}_k)$ using $q(\alpha)$, where $\vec{d}_k$ is the search direction and $\vec{x}_k$ the current starting point, such that

\begin{align} \varphi(0) &= f(\vec{x}_k) \\ \varphi'(0) &= \nabla f(\vec{x}_k)’ \vec{d}_k \end{align}

then the above formulas simplify to

\begin{align} a &= – \frac{\varphi(0) – \varphi(\alpha) + \alpha \varphi'(0)}{\alpha^2} \\ b &= \frac{\alpha^2 \varphi'(\alpha)}{\alpha^2} \\ c &= \frac{\alpha^2 \varphi(0)}{\alpha^2} \end{align}

and, more importantly, the local (approximated) minimizer at $\alpha_{min}$ simplifies to

\begin{align} \alpha_{min} &= \frac{1}{2} \frac{\alpha^2 \varphi'(0)}{\varphi(0)-\varphi(\alpha)+\alpha\varphi'(0)} \end{align}

If $q(\alpha)$ is required to be strongly convex, then we’ll observe that

\begin{align} q”(\alpha) &= 2a \overset{!}{\succeq} m \end{align}

for an $m > 0$, giving us that $a$ must be greater than zero (or $\epsilon$, for that matter), which is a trivial check. The following picture visualizes that this is indeed the case:

Convexity of a parabola for different highest-order coefficients a with positive b (top), zero b (middle) and negative b (bottom). Lowest-order coefficient c is left out for brevity.