Commodities in Black-Litterman

For those people trying to use the Black Litterman to model asset classes other than equities and bonds, the challenge of commodities is an interesting one. How does one figure out the market capitalization of commodities, and what is the relationship between the ownership of reserves by natural resource companies (thus included in their market capitalization), and how do futures which might exceed the size of the spot market by some multiple fit in.

Tom Idzorek wrote a paper which covers some of these topics. I've seen a few more around the internet and I'll try to start putting them up here and also on the blacklitterman.org website.

Updated blacklitterman.org

I updated the blacklitterman.org website with a definitely working Scilab script for the He and Litterman paper, and hopefully working Matlab scripts for the same thing. I should get a chance to debug the Matlab scripts in the next day. It is one of the most common queries on my website, looking for Matlab Black-Litterman scripts.

I've been having a lot of good discussions with co-workers about how one might structure a tool for Black-Litterman work. It's helping me continue to get a better handle on the various ideas and how they all interrelate. At this point it seems the math is the easy part.

I'm hoping to try and pull this together and update the paper since it's been almost 6 months since the last update. Not too many new things to put in there, but maybe some more flesh on the bones.

Also trying to clean things up in the website html a bit, finally moving to a dreamweaver template for the site so it's easier to make site wide changes.

Random Matrix Theory

Between Rosenow, et al and Potter, et al I've figured out a few ways people suggest to 'clean' a 'noisy' correlation matrix. Basically take the largest eigenvalues and either average the rest, or set the rest to 0 depending, reconstitute the matrix and set the diagonal to 1. By setting the noisy eigenvalues to 0, you have a sort of shrinkage towards uncorrelated assets which is interesting. I haven't quite thought through what the implication of that is on a mean-variance optimization.

Asset Allocation Technology

This blog is really discussing the technology of Asset Allocation, and hopefully some on the topic of the asset allocation process. I will not be discussing potential asset allocations.
Most recently I began to work on building some programs to perform portfolio optimization using CVaR, Conditional Value at Risk as the objective function. This requires a linear optimization algorithm, which I don't currently have in my toolbox. Someplace along the way of building this linear optimizer I ran into the topic of Random Matrix Theory.
Econophysics, it always draws me in. I think it's because of my Physics background, and how appealing their approaches are. Entropy is another Econophysics concept that I've spent some time looking at, but it hasn't added up to much in terms of portfolio optimization yet.
As far as Random Matrix Theory it also seems interesting, and it's not to difficult to figure out which part of the correlation matrix appears to be noisy, but it is somewhat complex to put the updated correlation matrix back together. While there are many papers on the topic, they don't usually provide a lot of information on re-assembling the correlation matrix. I'm hoping to make some progress in this direction at some point myself.
In the meantime I'm back working on my linear least squares optimizer. I was trying to work with the augmented form of the equations, but so far I have some errors in my math as my matrices are singular. Ouch.

CVaR

I decided to take a look at CVaR as an objective function. It has some desirable features, it can be solved using simulation and a good linear solver for sparse problems. This was all good in theory, but my Java simplex solver just isn't up to the task, so it's required me to go back to square 1 and build myself a new linear interior points solver. Given that I built a reasonable interior points solution for dense quadratic problems, I'm hoping it's not too much of a stretch to build a reasonable one for sparse linear problems.

lp_solve seems like a good tool in the problem space, but it's C so integration is a bit of work and I'd rather have a pure java solution in the short term even if it's not quite so good. I am not trying to solve general purpose problems, just CVaR.

I'm hoping to get this up on the site in the next few weeks, it depends on how much time I have for my hobby.