## Michael Jordan on the Top Open Problems in Bayesian Statistics

A colleague sent along the March 2011 issue of the ISBA Bulletin in which Michael Jordan lists the top five open problems in Bayesian statistics. The doyen at the intersection of ML/statistics/Bayesian stuff polled his peers and came up with this list:

**Model selection and hypothesis selection**: Given data, how do we select from a set of potential models? How can we be certain that our selection was correct?**Computation and statistics**: When MCMC is too slow/infeasible, what do we do? More importantly: “Several respondents asked for a more thorough integration of computational science and statistical science, noting that the set of inferences that one can reach in any given situation are jointly a function of the model, the prior, the data and the computational resources, and wishing for more explicit management of the tradeoffs among these quantities.”**Bayesian/frequentist relationships**: This is getting at the situation for high dimensional models when a subjective prior is hard to specify, and a simple prior is misleading. Can we then “…give up some Bayesian coherence in return for some of the advantages of the frequentist paradigm, including simplicity of implementation and computational tractability”?**Priors**: No surprise here. One resondent had a fascinating comment: when we want to model data that arises from human behavior*and*human beliefs, then we would expect/desire effects on both the prior and likelihood. Then what do we do?**Nonparametrics and semi-parametrics**: What are the classes of problems for which NP Bayes methods are appropriate/”worth the trouble”? In NLP, clustering (i.e., using DP priors) is certainly an area in which nonparametrics have been successful.

## Grad School Principles to Live By

Having been admitted to the Ph.D program for Fall 2011, I’ve finally allowed myself to think in broader strokes about the next few years. I’ve also been talking to anyone who will listen about how to structure a “successful” program. It turns out that success is specific to the individual, although a reasonable starting point is to settle the industry v. academia career question. Of course there are commonalities between the two trajectories: you must pass quals, you must write papers, you must complete a thesis. However, the end-goal should shape the nature of the research and how time is invested. Prospective professors should focus solely on a high-impact scientific contribution. It seems that networking opportunities, invitations to speak, and so on follow from doing respected work. Make a name for yourself.

The industry track is either easier or harder, depending on the time scale. If you want to spend your career in an industrial research lab, then you can do more practical grad school research, network at conferences, and do internships. However, if an executive role is the final objective, then it might be wise to include a b-school certificate, to collaborate with a company during grad school, and to take some non-technical courses.

Presently, I am working through these alternatives.

In my conversations with both peers and professors, a core set of principles has emerged that seem reliable irrespective of the career path. They are:

**Take classes, but no more than one per quarter**: Stanford’s Ph.D program does not require coursework. As I did not have a thorough undergraduate training in my research field, this aspect of the system works against me. I need to take courses, but the challenge is to limit the interruption to my research.**Work on two projects simultaneously, but no more**: It is refreshing to switch projects when a barrier is encountered. The mind benefits. However, life is short; two is enough.**There is no substitute for good research**: It’s like cash in your checking account.**Mind the technical foundation**: Every project has an engineering component. I find that my mathematical and analytical skills erode when I spend too much time programming. A good way to workout those muscles regularly is to write “squibs,” or short pieces on a specific proof, idea, technique, etc. Writing is equivalent to implementing. If you can write it down, you know it.

## Bad Start to Diving Season

Although those of us who dive in Monterey regularly know that the ocean there has no seasons–it is always unpredictable–many more people seem to dive in the spring and summer. As a consequence, the dive boats run more charters, the dive shops stay open later, and it’s even harder to book a reservation at Pt. Lobos. This season has not gotten off to an auspicious start: three deaths in the last ten days, and two other divers stranded at Monastery.

Father and daughter rescued off Monastery