Latest Comments
Search

[Good Idea?] Book on automating P2P Lending

I'm considering writing a book on automating P2P lending (in Python!) on self-publishing sites like leanpub and lulu, or Amazon KDP. Good idea? Terrible idea?

Meh. Who am I kidding. I'm going to just do it. ;)

You're onto something when...

Trying to figure out: 1) how to scale the operation, 2) decide whether or not to use leverage and, most importantly, 3) does it make financial sense to use Amazon EC2 compute instances?

Combining Multiple Model Probabilities

Here's another area of work for modeling Prosper loans. How do I combine the probabilities of multiple models? I started with the brain-dead approach of using a convex combination of probabilities: 

This can be interpretated as the coefficients directly proportional to the amount of information each model has. The optimal weights can be estimated from training data. However I am wondering if I can improve this approach and, of course, the Google is all powerful. Time for some more bed-time reading.

Risk Analysis, Vol. 19, No. 2, 1999. Combining Probability Distributions From Experts in Risk Analysis. R. T. Clemen and R. L. Winkler.

Technical Report no. 543, Department of Statistics, University of Washington, Oct 2008. Combining Probability Forecasts. R. Ranjan and T. Gneiting.

Prosper P2P Loan Model

My latest side project has been working on modeling Prosper P2P loans. Features of the data like credit grade, income, etc. are all readily available for import (it's like they actually want you to use this data!). All of the data is imported in MySQL using a script, and then sliced and diced using R.

I created a runtime model, which is an ensemble of various machine learning models (think Netflix prize grand prize solution). The runtime model runs on a Raspberry Pi and uses the Prosper API to place bids automatically.

Mental health is important

Since my 20s, I have been meditating on a semi-regular basis; having attended meditation classes, retreats and read multiple books on the subject.

Why meditate?

The brain is not idempotent: as the mind thinks, the brain changes too. For example, London taxi drivers with years of navigating the twisty and confusing streets of road developed a larger hippocampus since that part of the brain is used for visual-spatial memories [Maguire et al 2006]. There is increasing evidence that meditation improves cognition [Lazar et al 2006] [Luders et al 2011]

Even after just a few days of training and practice, participants with no prior meditation experienced reduced fatigue and anxiety, together with increased mindfulness [Zeidan et al 2010]. Moreover, brief mindfulness training significantly improved visuo-spatial processing, working memory, and executive functioning.

Meditation trains your mind to treat negative situations with equanimity, which results in a natural state of calm and happiness.

Readings

Buddha’s Brain: Neuroplasticity and Meditation (online article)

Buddha’s Brain: The Practical Neuroscience of Happiness, Love, and Wisdom (book)

Can Meditation Make You Smarter? (online article)

Evidence Builds That Meditation Strengthens the Brain (online article)

The Unique Brain Anatomy of Meditation Practitioners: Alterations in Cortical Gyrification (online article)

What is meditation?

In this article, I use the word meditation to refer to Concentration (or Samatha) meditation and Insight (or Vipassana) meditation. There are multiple meditation methods, but the forms that I have practiced extensively with are breath meditation and body scanning.

Samatha is really simple, and is used in the Theravada tradition as the preliminary meditation technique to build a foundation to embark on Vipassana meditation. The practice I have learnt and practiced most extensively is the the breath meditation. Follow the sensations of the breath as it flows in through and nostrils, keeping your attention at the tip of the nostrils. Your mind may wander, but keep patiently returning to the breath.

It is not necessary to develop concentration to the point of excluding everything else except the breath. The purpose of this practice to to allow you to rein in the mind, and allow you to ntoice the workings of the mind. The entire process of gathering your attention, noticing your breath, noticing that your mind has wandered, and re-focusing you attention develops mindfulness, patience and concentration. Deep and sustained practice in Samatha allow you to reach various states of consciousness called Jhanas.

Vipassana are meditation techniques to develop mindfulness and is used to become aware of the impermanence of everything that exists. Vipassana provides you with the guidance on how to see clearly into the nature of the mind, which claims to lead to liberation. I am practicing the body sweeping meditation to develop mindfulness.

Readings

Mindfulness in Plain English (book)

Search Inside Yourself: The Unexpected Path to Achieving Success, Happiness (and World Peace) (book)

Focused and Fearless: A Meditator's Guide to States of Deep Joy, Calm, and Clarity (book)

Wisdom Wide and Deep: A Practical Handbook for Mastering Jhana and Vipassana (book)

Mastering the Core Teachings of the Buddha: An Unusually Hardcore Dharma Book: An Unusually Hardcore Dharma Book (book)

How to practice?

My practice in my 20s has been spotty: I would diligently meditate an hour every day for a month, and then lapse into meditating once a week (maybe) for 10 minutes. Even the type of meditation would vary: one month it would be concentration meditation like breath meditation on the tip of the nostril, the next it would be Vipassana or Insight meditation by scanning the body, and yet another it would be loving kindness meditation. Heck, I even tried meditating for a period of time on a burning flame.

A few years back, I realized that this inconsistent practice meant that I was not incorporating the goal of meditation by integrating the practice into my daily life. I was riding the whims of my monkey mind and siloing the practice into a separate aspect of my life.

Unfortunately, meditation as an activity is so mind-numbingly boring (initially). Some motivation techniques suggested can be using the carrot or the stick. The carrot includes the benefits of meditation, both secular (mental wellness) and spiritual (enlightenment). The stick might be to practice “death” meditation to become aware of our morality.

One mental hack that I have learnt over the years is to institute a habit to perform a desirable activity. The label “habit” is very persuasive, especially since you are engaging in an initially undesirable activity. I currently meditate for at least 30 minutes before going to bed, and meditate on weekends for an hour. You only have a finite amount of willpower each day, so forming a habit makes it easier to carry out. How long will it take to form this habit? Research suggests that forming a meditative practice will take over two months of daily repetition before the behavior becomes a habit (and skipping single days is not detrimental in the long-term). [Lally et al 2009]

Readings

Willpower (book)

Habits: How They Form And How To Break Them (online article)

The Power of Habit (book)

San Francisco Sourdough Bread

I have been baking bread to bring to work and someone asked for the recipe, which is modified based on ideas from Tartine Bread and The Bread Baker's Apprentice books.

Wild Yeast Levain

  • 1/4 cup sourdough starter
  • 1 and 3/4 cups unbleached bread or all purpose flour
  • 1/2 cup lukewarm water

Mix the ingredients by hand for a few minutes. The mix should be slightly sticky but doughlike. Transfer the starter to a work surface and knead for a minute. Transfer to a bowl with oiled surface and leave it until it has doubled in size. The temperature in my apartment falls to about 60F at night and I just leave it out on my countertop overnight. To retard the growth, you can store the levain in the refrigerator for a few days.

Dough (makes two loaves)

  • all of levain
  • 1 and 3/4 cup lukewarm water
  • 4 and 1/2 cup of unbleached bread or all purpose flour
  • 3 teaspoons of coarse salt
  • [optional] 4 tablespoons of potato flour
  • [optional] nuts and seeds, about 4 tablespoons

Sift the flour and salt, then add in the levain and water. Mix by hand for a few minutes until the dough coarsely holds together and then let it rest for 5 minutes. By giving the gluten time to form the bonds, time helps me strengthen the dough and makes the next mixing and kneading steps easier.

After a short rest, continue mixing, adding flour and/or water until the dough is soft and only slightly sticky. This is also when I add the nuts and seeds. Then transfer to a work surface and knead for a few minutes.

Use stretch and fold technique to develop the dough. Let the dough rest for 10 to 30 minutes in a bowl or container, and then stretch and fold the dough. Repeat one or two more times.

After last stretch and fold, transfer to a lightly oiled bowl which is big enough for the dough to double in, and cover. Leave the bowl out for 1 to 2 hours and then leave in the refrigerator until baking day.

Baking

I bake in the morning, so I take the dough out the night before. I leave it for 2 hours to warm to room temperature, and then I split the dough into halves. I then shape the dough and leave them overnight (about 10 hours) to proof.

In the morning, I preheat the oven to 500F for 30 minutes. The oven has two pizza stones and a metal pan. So even though the oven might say it's reached the right temperature, I do not proceed until the baking stones are warmed up too.

Turn the shaped dough onto a pizza peel, and then score it ([boule] and [batard]) just before placing it in the oven. Repeat for the other dough. Before closing the door I add one cup of hot water to the metal pan.

After one minute, I spray water to the side of the oven, and then again the next minute. I then lower the oven temperature to 450F. After 15 minutes I rotate the loaves so that the baking is even. I leave the loaves in for another 20 to 30 minutes.

Bread with sourdough starter

Be Credible by being Bayesian

Recently at work we encountered a problem with the "standard" way of computing confidence intervals for a Binomial proportion using the Normal approximation. That is,
At small sample sizes and high sample proportions, this is too conservative. For example, with a sample size of 10, the confidence intervals for a sample proportion of 0.8 at 95% confidence level, the confidence interval based on the Normal approximation is [0.55, 1].

We can easily see that this confidence interval is too wide as it includes p = 1.0, but we already saw 2 negative examples, which flies in the face of the evidence that p = 1.0 should be on the table.

There are many alternative confidence interval computation methods available, but one particular approach which I prefer is Bayesian: Use a Beta prior disitribution and use that to obtain a posterior distribution (Beta-Binomial). For example, using the Uniform (Uninformed) prior distribution Beta(1,1), the confidence interval (or "credible interval") is the much more plausible [0.480.94].

One of the nice properties of this approach is the subjective knowledge that is inherently available as input to the model. The hyperparameters alpha and beta can be interpreted as alpha - 1 is the number of successes and beta - 1 is the number of failures. So if we strongly believe a priori that the sample proportion is around 0.9, then we could use the Beta(10, 2) as the prior distribution instead, which gives the credible interval of [0.64, 0.95].

The Ph.D. Grind

This is great. Highly recommended reading for current and potential Ph.D. candidates.

Philip Guo wrote a memoir documenting his Ph.D. experiences and released it online for free (http://pgbovine.net/PhD-memoir.htm).

Four fingers back at me

So here's a change in tack since my ancecdote on "PhD jobs" (aka good technical jobs in Singapore) now turns up within the top ten results for [phd jobs singapore].

Quite simply, it is my sincere belief that there needs to be more companies where there is excellent engineer culture (*). This creates a cycle where the brightest and best will gravitate towards these areas of study. Merely having one or two successful company is not enough, you want a garden of them. Creative technology anyone?

As it turns out, Singapore is starting to have some traction as a start up hub in Southeast Asia. So entrepreneurs - make your millions and billions, and play your part in making software engineering (and technical jobs) a sexy career choice in Singapore!

p.s., I'm very contented in Google; but feel free to pitch to me if you want my opinion on getting stuff done.

(*) You want to foster a culture where innovating is the norm, and the Silicon Valley model has worked well so I am limited by this mindset, but it is certainly true that there are other ways to achieve this.

Do I believe in this list?

According to The Times Higher Education ranking for engineering and technology universities, the National University of Singapore is ranked 19th versus the University of Waterloo which is ranked 48th.

Pssssh. Please.