SIA - Smart Investment Advisor: Python optimization

Challenge description

• Convert a set of Excel functions into optimised Python
• Targeting Python3 and Jupyter Notebook (for easy review)

Project background

We are building a Smart Investment Advisor platform that empowers users of financial/brokerage sites to back-test, find new ideas for, optimize and trade their portfolios, based on proven theories pioneered by Nobel Prize-winning economists. Traditional financial/brokerage sites only allow users to see price charts and other data of individual stocks, but do not provide a holistic view of the portfolio

This challenge will be part of the platform, used to optimise a user's portfolio allocations.

Technology

For this challenge, we will target:

• Numpy or Pandas
• Python3
• Jupyter Notebook

You can assume that reviewers will have Python3, Jupyter Notebook, and the most recent versions of Numpy and Pandas installed.  Please also specify the versions you tested in your README to avoid any conflicts.
Your submission should be a detailed Jupyter Notebook with specific functions defined for each of the steps listed below (and described in the XLS under `STEPS`)
The reviewers should be able to load your Jupyter Notebook and run through the steps, getting output similar to the XLS.

PyPortfolioOpt

The client has pointed us to the Python library `PyPortfolioOpt` as an example of something that could help guide development.  *NOTE* - We won't use this library in our implementation, but you can use it to help understand the requirements and the algorithms needed, especially the Ledoit Wolf part of the covariance calculation.

Main requirements

In the forum will be provided an optimiser spreadsheet.  This spreadsheet includes 10 steps that we will convert from Excel into Python on a Jupyter notebook.
NOTE - Calc_EF is likely the most complex part of the requirements.  Please ensure you budget enough time to fully understand and implement that algorithm.

Step 1 -- Store_User_Port

The first step is to load the user's portfolio.  For the purposes of this challenge, we can expect this to be in a CSV format, at the discretion of the submitter.  Please provide a reasonable sample file and the functionality for loading this into Numpy / Pandas in the Jupyter Notebook.

Step 2 -- Get_Price_Data

Similar to the first step, this function will load pricing data from a CSV file.  Please take the data on the `PRICE` sheet, convert it to a usable CSV format, and use that data.

Step 3 -- Calc_Rtn

This function takes the pricing data loaded above and calculates the expected return for each stock.  We just want to make sure this is efficiently implemented.

Step 4 -- Store_User_Exp_Rtn

This doesn't necessarily have to be a function in the Python but can be a set of variables that the reviewers and client can use to set the expected return parameters, similar to Get_Wgt_Const below.

Step 5 -- Calc_VarCov

Using the data from Get_Price_Data, this will calculate the covariance matrix for the symbols.  Note that the XLS refers to Ledoit Wolf shrinkage - that shrinkage implementation is not in scope for this particular challenge.  That will be added later.

Step 6 -- Get_Wgt_Const

This can just be a set of variables used to tweak the stock weight constraints by the user (minimum and maximum)
 
From the client:  "Also, we should have the capability of ADDING a weight constraint (to limit min weight and max weight) for one or more securities if the portfolio, if the user decides that certain securities are NOT subject to the default weight constraints. Please note that the optimization process itself can also be iterative because the user may choose a point on EF, and then adjust the stock weight (and/or min/max weight for particular stock) and/or expected return assumption,then re-run the optimization process again."

Step 7 -- Calc_EF

This function will do the calculations for different returns and volatility at various settings.  Note that this is a complex function, taking into account the user variables.  A separate `Solver Macro` XLS has been provided that can be used as a template and reference for the algorithm needed here.  Note that PyPortfolioOpt appears to use the solver from Scipy, but there could be better options available.

Step 8 -- Clean_Weight

As described in the XLS, this just rounds the results to reasonable values.

Step 9 -- Plot_Port

This function will be implemented to plot the EF results.  This can be done using various plotting packages that work with Jupyter Notebook.  Matplotlib or Seaborn would both be fine.

Backtest

The solver macro file has tabs for backtesting - please ensure we have simple backtest functionality in the Jupyter Notebook as well.  We want to mimic what's shown in the solver macro file.

Optimization

The goal for this challenge is to convert the functionality, but *also* ensure that it's efficiently implemented.  The most efficient implementation (as determined by the copilot, reviewers, and client) will receive a $400 bonus.
Things to consider include custom functions using Numba and it's JIT functionality, multithreading, CUDA, etc...

Testing

Please ensure the results from your implementation matches the spreadsheets and the sample data there.  It's important that our algorithms are accurate.

Code access

The existing optimization spreadsheet and the solver macro will be provided in the forum.

Deployment documentation

Because we are using Jupyter Notebook, it's fine to include the README and validation documentation as comments in the notebook, as long as they are complete and helpful to reviewers and the client.  You are also welcome to submit a README and validation document as normal.

No video is required for this challenge

What to submit

• Jupyter Notebook file
• CSV files with sample data
• (optional) README and Validation files

Final Submission Guidelines

Please see above