Physicist, Startup Founder, Blogger, Dad

Wednesday, March 14, 2018

Stephen Hawking (1942-2018)

Roger Penrose writes in the Guardian, providing a scientifically precise summary of Hawking's accomplishments as a physicist (worth reading in full at the link). Penrose and Hawking collaborated to produce important singularity theorems in general relativity in the late 1960s.

Here is a nice BBC feature: A Brief History of Stephen Hawking. The photo above was taken at Hawking's Oxford graduation in 1962.
Stephen Hawking – obituary by Roger Penrose

... This radiation coming from black holes that Hawking predicted is now, very appropriately, referred to as Hawking radiation. For any black hole that is expected to arise in normal astrophysical processes, however, the Hawking radiation would be exceedingly tiny, and certainly unobservable directly by any techniques known today. But he argued that very tiny black holes could have been produced in the big bang itself, and the Hawking radiation from such holes would build up into a final explosion that might be observed. There appears to be no evidence for such explosions, showing that the big bang was not so accommodating as Hawking wished, and this was a great disappointment to him.

These achievements were certainly important on the theoretical side. They established the theory of black-hole thermodynamics: by combining the procedures of quantum (field) theory with those of general relativity, Hawking established that it is necessary also to bring in a third subject, thermodynamics. They are generally regarded as Hawking’s greatest contributions. That they have deep implications for future theories of fundamental physics is undeniable, but the detailed nature of these implications is still a matter of much heated debate.

... He also provided reasons for suspecting that the very rules of quantum mechanics might need modification, a viewpoint that he seemed originally to favour. But later (unfortunately, in my own opinion) he came to a different view, and at the Dublin international conference on gravity in July 2004, he publicly announced a change of mind (thereby conceding a bet with the Caltech physicist John Preskill) concerning his originally predicted “information loss” inside black holes.
Notwithstanding Hawking's premature 2004 capitulation to Preskill, information loss in black hole evaporation remains an open question in fundamental physics, nearly a half century after Hawking first recognized the problem in 1975. I read this paper as a graduate student, but with little understanding. I am embarrassed to say that I did not know a single person (student or faculty member) at Berkeley at the time (late 1980s) who was familiar with Hawking's arguments and who appreciated the deep implications of the results. This was true of most of theoretical physics -- despite the fact that even Hawking's popular book A Brief History of Time (1988) gives a simple version of the paradox. The importance of Hawking's observation only became clear to the broader community somewhat later, perhaps largely due to people like John Preskill and Lenny Susskind.

I have only two minor recollections to share about Hawking. The first, from my undergraduate days, is really more about Gell-Mann: Gell-Mann, Feynman, Hawking. The second is from a small meeting on the black hole information problem, at Institut Henri Poincare in Paris in 2008. (My slides.) At the conference dinner I helped to carry Hawking and his motorized chair -- very heavy! -- into a fancy Paris restaurant (which are not, by and large, handicapped accessible). Over dinner I met Hawking's engineer -- the man who maintained the chair and its computer voice / controller system. He traveled everywhere with Hawking's entourage and had many interesting stories to tell. For example, Hawking's computer system was quite antiquated but he refused to upgrade to something more advanced because he had grown used to it. The entourage required to keep Hawking going was rather large (nurses, engineer, driver, spouse), expensive, and, as you can imagine, had its own internal dramas.

Saturday, March 10, 2018

Risk, Uncertainty, and Heuristics

Risk = space of outcomes and probabilities are known. Uncertainty = probabilities not known, and even space of possibilities may not be known. Heuristic rules are contrasted with algorithms like maximization of expected utility.

See also Bounded Cognition and Risk, Ambiguity, and Decision (Ellsberg).

Here's a well-known 2007 paper by Gigerenzer et al.
Helping Doctors and Patients Make Sense of Health Statistics

Gigerenzer G1, Gaissmaier W2, Kurz-Milcke E2, Schwartz LM3, Woloshin S3.

Many doctors, patients, journalists, and politicians alike do not understand what health statistics mean or draw wrong conclusions without noticing. Collective statistical illiteracy refers to the widespread inability to understand the meaning of numbers. For instance, many citizens are unaware that higher survival rates with cancer screening do not imply longer life, or that the statement that mammography screening reduces the risk of dying from breast cancer by 25% in fact means that 1 less woman out of 1,000 will die of the disease. We provide evidence that statistical illiteracy (a) is common to patients, journalists, and physicians; (b) is created by nontransparent framing of information that is sometimes an unintentional result of lack of understanding but can also be a result of intentional efforts to manipulate or persuade people; and (c) can have serious consequences for health. The causes of statistical illiteracy should not be attributed to cognitive biases alone, but to the emotional nature of the doctor-patient relationship and conflicts of interest in the healthcare system. The classic doctor-patient relation is based on (the physician's) paternalism and (the patient's) trust in authority, which make statistical literacy seem unnecessary; so does the traditional combination of determinism (physicians who seek causes, not chances) and the illusion of certainty (patients who seek certainty when there is none). We show that information pamphlets, Web sites, leaflets distributed to doctors by the pharmaceutical industry, and even medical journals often report evidence in nontransparent forms that suggest big benefits of featured interventions and small harms. Without understanding the numbers involved, the public is susceptible to political and commercial manipulation of their anxieties and hopes, which undermines the goals of informed consent and shared decision making. What can be done? We discuss the importance of teaching statistical thinking and transparent representations in primary and secondary education as well as in medical school. Yet this requires familiarizing children early on with the concept of probability and teaching statistical literacy as the art of solving real-world problems rather than applying formulas to toy problems about coins and dice. A major precondition for statistical literacy is transparent risk communication. We recommend using frequency statements instead of single-event probabilities, absolute risks instead of relative risks, mortality rates instead of survival rates, and natural frequencies instead of conditional probabilities. Psychological research on transparent visual and numerical forms of risk communication, as well as training of physicians in their use, is called for. Statistical literacy is a necessary precondition for an educated citizenship in a technological democracy. Understanding risks and asking critical questions can also shape the emotional climate in a society so that hopes and anxieties are no longer as easily manipulated from outside and citizens can develop a better-informed and more relaxed attitude toward their health.

Wednesday, March 07, 2018

The Ballad of Bedbug Eddie and the Golden Rule

This is a bedtime story I made up for my kids when they were small. See also Isabel and the dwarf king.
Once upon a time, there was a tiny bedbug named Eddie, who was no bigger than a sesame seed. Like all bedbugs, Eddie lived by eating the blood of humans. Every night he crawled out of the bedding and bit his sleeping victim.

One night a strange idea entered Eddie's mind. Are there little bugs that bite me when I sleep? he wondered. That would be terrible! (Little did Eddie know that there was a much smaller bug named Mini who lived in his left antenna, and who drank his blood! But that is another story...)

Suddenly, Eddie had an inspiration. It was wrong to bite other people and drink their blood. If I don't like it, he thought, I shouldn't do it to other people!

From that moment on, Eddie resolved to never bite another creature. He would have to find a source of food other than blood!

Eddie lay in his bedding nest and wondered what he would do next. He had never eaten any other kind of food. He realized that to survive, he would have to search out a new kind of meal.

When the sun came up, Eddie decided he should leave his bed in search of food. He wandered through the giant house, with its fuzzy carpeting and enormous potted plants. Finally he came upon the cool, smooth floor of the kitchen. Smelling something edible, he continued toward the breakfast table.

Soon enough, he encountered the biggest chunk of food he had ever seen. It was a hundred times bigger than Eddie, and smelled of peanut butter -- it was a crumb of toast! Then Eddie realized the entire floor under the table was covered with crumbs -- bread, cracker, muffin, even fruit and vegetable crumbs!

Eddie jumped onto the peanut butter toast crumb and started to eat. He was very hungry after missing his usual midnight meal. He ate until he was very full. It took some getting used to peanut butter -- not his usual blood meal! But he would manage.

Suddenly, a huge crumb fell from the sky and almost crushed Eddie. He barely managed to jump out of the way of the huge block of cereal, wet with milk. Looking up, he saw a giant figure on a chair, who was spraying crumbs all around as he gobbled up his breakfast.

The Crumb King! exclaimed Eddie. The Crumb King provides us with sustenance!

Hello Crumb King, shouted Eddie. Look out below! You almost crushed me with that cereal! he yelled.

Between crunches of cereal, Max heard a tiny voice from below. Surprised, he looked down at the small black dot, no bigger than a sesame seed. Are you a bug? he asked.

I am bedbug Eddie! responded Eddie. Don't crush me with crumbs! he shouted.

From that day on, Eddie and Max were great friends.

Eddie became a vegetarian and devoted his life to teaching the Golden Rule: "Do unto others as you would have them do unto you.” (Matthew 7:12)

Better to be Lucky than Good?

The arXiv paper below looks at stochastic dynamical models that can transform initial (e.g., Gaussian) talent distributions into power law outcomes (e.g., observed wealth distributions in modern societies). While the models themselves may not be entirely realistic, they illustrate the potentially large role of luck relative to ability in real life outcomes.

We're used to seeing correlations reported, often between variables that have been standardized so that both are normally distributed. I've written about this many times in the past: Success, Ability, and All That , Success vs Ability.

But wealth typically follows a power law distribution:

Of course, it might be the case that better measurements would uncover a power law distribution of individual talents. But it's far more plausible to me that random fluctuations + nonlinear amplifications transform, over time, normally distributed talents into power law outcomes.

Talent vs Luck: the role of randomness in success and failure

The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, smartness, efforts, willfulness, hard work or risk taking. Sometimes, we are willing to admit that a certain degree of luck could also play a role in achieving significant material success. But, as a matter of fact, it is rather common to underestimate the importance of external forces in individual successful stories. It is very well known that intelligence (or, more in general, talent and personal qualities) exhibits a Gaussian distribution among the population, whereas the distribution of wealth - often considered a proxy of success - follows typically a power law (Pareto law), with a large majority of poor people and a very small number of billionaires. Such a discrepancy between a Normal distribution of inputs, with a typical scale (the average talent or intelligence), and the scale invariant distribution of outputs, suggests that some hidden ingredient is at work behind the scenes. In this paper, with the help of a very simple agent-based toy model, we suggest that such an ingredient is just randomness. In particular, we show that, if it is true that some degree of talent is necessary to be successful in life, almost never the most talented people reach the highest peaks of success, being overtaken by mediocre but sensibly luckier individuals. As to our knowledge, this counterintuitive result - although implicitly suggested between the lines in a vast literature - is quantified here for the first time. It sheds new light on the effectiveness of assessing merit on the basis of the reached level of success and underlines the risks of distributing excessive honors or resources to people who, at the end of the day, could have been simply luckier than others. With the help of this model, several policy hypotheses are also addressed and compared to show the most efficient strategies for public funding of research in order to improve meritocracy, diversity and innovation.
Here is a specific example of random fluctuations and nonlinear amplification:
Nonlinearity and Noisy Outcomes: ... The researchers placed a number of songs online and asked volunteers to rate them. One group rated them without seeing others' opinions. In a number of "worlds" the raters were allowed to see the opinions of others in their world. Unsurprisingly, the interactive worlds exhibited large fluctuations, in which songs judged as mediocre by isolated listeners rose on the basis of small initial fluctuations in their ratings (e.g., in a particular world, the first 10 raters may have all liked an otherwise mediocre song, and subsequent listeners were influenced by this, leading to a positive feedback loop).

It isn't hard to think of a number of other contexts where this effect plays out. Think of the careers of two otherwise identical competitors (e.g., in science, business, academia). The one who enjoys an intial positive fluctuation may be carried along far beyond their competitor, for no reason of superior merit. The effect also appears in competing technologies or brands or fashion trends.

If outcomes are so noisy, then successful prediction is more a matter of luck than skill. The successful predictor is not necessarily a better judge of intrinsic quality, since quality is swamped by random fluctuations that are amplified nonlinearly. This picture undermines the rationale for the high compensation awarded to certain CEOs, studio and recording executives, even portfolio managers. ...

Saturday, March 03, 2018

Big Tech compensation in 2018

I don't work in Big Tech so I don't know whether his numbers are realistic. If they are realistic, then I'd say careers in Big Tech (for someone with the ability to do high level software work) dominate all the other (risk-adjusted) options right now. This includes finance, startups, etc.

No wonder the cost of living in the bay area is starting to rival Manhattan!

Anyone care to comment?

Meanwhile, in the low-skill part of the economy:
The Economics of Ride-Hailing: Driver Revenue, Expenses and Taxes

MIT Center for Energy and Environmental Policy Research

We perform a detailed analysis of Uber and Lyft ride-hailing driver economics by pairing results from a survey of over 1100 drivers with detailed vehicle cost information. Results show that per hour worked, median profit from driving is $3.37/hour before taxes, and 74% of drivers earn less than the minimum wage in their state. 30% of drivers are actually losing money once vehicle expenses are included. On a per-mile basis, median gross driver revenue is $0.59/mile but vehicle operating expenses reduce real driver profit to a median of $0.29/mile. For tax purposes the $0.54/mile standard mileage deduction in 2016 means that nearly half of drivers can declare a loss on their taxes. If drivers are fully able to capitalize on these losses for tax purposes, 73.5% of an estimated U.S. market $4.8B in annual ride-hailing driver profit is untaxed.
Note Uber disputes this result and claims the low hourly result is due in part to the researchers misinterpreting one of the survey questions. Uber's analysis puts the hourly compensation at ~$15.

How NSA Tracks You (Bill Binney)

Anyone who is paying attention knows that the Obama FBI/DOJ used massive government surveillance powers against the Trump team during and after the election. A FISA warrant on Carter Page (and Manafort and others?) was likely used to mine stored communications of other Trump team members. Hundreds of "mysterious" unmasking requests by Susan Rice, Samantha Powers, etc. were probably used to identify US individuals captured in this data.

I think it's entirely possible that Obama et al. thought they were doing the right (moral, patriotic) thing -- they really thought that Trump might be colluding with the Russians. But as a civil libertarian and rule of law kind of guy I want to see it all come to light. I have been against this kind of thing since GWB was president -- see this post from 2005!

My guess is that NSA is intercepting and storing big chunks of, perhaps almost all, US email traffic. They're getting almost all metadata from email and phone traffic, possibly much of the actual voice traffic converted to text using voice recognition. This used to be searchable only by a limited number of NSA people (although that number grew a lot over the years; see 2013 article and LOVEINT below), but now available to many different "intel" agencies in the government thanks to Obama.

Situation in 2013: https://www.npr.org/templates/story/story.php?storyId=207195207

(Note Title 1 FISA warrant grants capability to look at all associates of target... like the whole Trump team.)

Obama changes in 2016: https://www.nytimes.com/2016/02/26/us/politics/obama-administration-set-to-expand-sharing-of-data-that-nsa-intercepts.html
NYT: "The new system would permit analysts at other intelligence agencies to obtain direct access to raw information from the N.S.A.’s surveillance to evaluate for themselves. If they pull out phone calls or email to use for their own agency’s work, they would apply the privacy protections masking innocent Americans’ information... ” HA HA HA I guess that's what all the UNmasking was about...
More on NSA capabilities: https://en.wikipedia.org/wiki/LOVEINT (think how broad their coverage has to be for spooks to be able to spy on their wife or girlfriend)

See also FISA, EO 12333, Bulk Collection, and All That.
Wikipedia: William Edward Binney[3] is a former highly placed intelligence official with the United States National Security Agency (NSA)[4] turned whistleblower who resigned on October 31, 2001, after more than 30 years with the agency.

He was a high-profile critic of his former employers during the George W. Bush administration, and later criticized the NSA's data collection policies during the Barack Obama administration. 
From the transcript of Binney's talk:
ways that they basically collect data
first it's they use the corporations
that run the fiber-optic lines and they
get them to allow them to put taps on
them and I'll show you some of the taps
where they are and and if that doesn't
work they use the foreign government to
go at their own telecommunications
companies to do the similar thing and if
that doesn't work they'll tap the line
anywhere they can get to it and they
won't even know it you know the
government's know that communications
companies will even though they're
tapped so that's how they get into it
then I get into fiber lines and this is
this is a the prism program ...

that was published
out of the Snowden material and they've
all focused on prism well prism is
really the the minor program I mean the
major program is upstream that's where
they have the fiber-optic taps on
hundreds of places around in the world
that's where they're collecting off the
fiber lined all the data and storing it
2016 FISC reprimand of Obama administration. The court learned in October 2016 that analysts at the National Security Agency were conducting prohibited database searches “with much greater frequency than had previously been disclosed to the court.” The forbidden queries were searches of Upstream Data using US-person identifiers. The report makes clear that as of early 2017 NSA Inspector General did not even have a good handle on all the ways that improper queries could be made to the system. (Imagine Snowden-like sys admins with a variety of tools that can be used to access raw data.) Proposed remedies to the situation circa-2016/17 do not inspire confidence (please read the FISC document).

Tuesday, February 27, 2018

MIT Technology Review: Genomic Prediction a 2018 Breakthrough Technology

From 10 Breakthrough Technologies 2018:
One day, babies will get DNA report cards at birth. These reports will offer predictions about their chances of suffering a heart attack or cancer, of getting hooked on tobacco, and of being smarter than average.

The science making these report cards possible has suddenly arrived, thanks to huge genetic studies—some involving more than a million people.

It turns out that most common diseases and many behaviors and traits, including intelligence, are a result of not one or a few genes but many acting in concert. Using the data from large ongoing genetic studies, scientists are creating what they call “polygenic risk scores.”

Though the new DNA tests offer probabilities, not diagnoses, they could greatly benefit medicine. For example, if women at high risk for breast cancer got more mammograms and those at low risk got fewer, those exams might catch more real cancers and set off fewer false alarms.

Pharmaceutical companies can also use the scores in clinical trials of preventive drugs for such illnesses as Alzheimer’s or heart disease. By picking volunteers who are more likely to get sick, they can more accurately test how well the drugs work.

The trouble is, the predictions are far from perfect. Who wants to know they might develop Alzheimer’s? What if someone with a low risk score for cancer puts off being screened, and then develops cancer anyway?

Polygenic scores are also controversial because they can predict any trait, not only diseases. For instance, they can now forecast about 10 percent of a person’s performance on IQ tests. As the scores improve, it’s likely that DNA IQ predictions will become routinely available. But how will parents and educators use that information? ...
Also from Technology Review: Forecasts of genetic fate just got a lot more accurate.

Friday, February 23, 2018

Kosen Judo and the origins of MMA

When I was in Japan in the mid-1990s almost no one outside of a small group of MMA fans had ever heard of BJJ or Gracie Jiujitsu. Sometimes when I went to a judo club to practice I would just explain that I was a "newaza specialist" (ground technique specialist) or even that I wanted to do Kosen-style judo.

The Imperial Universities that specialized in Kosen judo did so partially because they were nerds! One could become adept at newaza with less natural athleticism and less practice than was required to become a true tachiwaza (standing technique = dynamic throws) specialist. A relatively small amount of training in ground technique allows a fighter to completely dominate an untrained opponent. The Kosen competitors would simply drag their opponent to the mat without using any flashy throws or takedowns, and then submit or pin them. More video.

I cannot really tell from the video whether these Kosen practitioners have also adopted techniques from modern BJJ. I see some spider guard, but apparently that is an old Kosen style! Don't let the black belts fool you. In Japan you go from white to black belt directly, and 1st dan black belt just means you know the basic moves and are still very much a student. These guys in the video don't look all that advanced to me for the most part. (It's not easy to be admitted to Kyoto University, by the way.)

Here's a top-level Kosen guy. He's destroying those scrubs in Canada ;-)

Wikipedia: Kosen judo (高專柔道 Kōsen jūdō) is a variation of the Kodokan judo competitive ruleset that was developed and flourished at the kōtō senmon gakkō (高等専門学校)(kōsen (高專)) technical colleges in Japan in the first half of the twentieth century. Kosen judo's rules allow for greater emphasis of ne-waza (寝技, ground techniques) than typically takes place in competitive judo and it is sometimes regarded as a distinct style of judo.

Today, the term "kosen judo" is frequently used to refer to the competition ruleset associated with it that allows for extended ne-waza. Such competition rules are still used in the Nanatei Jūdō / Shichitei Jūdō (七帝柔道 Seven Imperials Judo) competitions held annually between the seven former Imperial universities. Similarly, there has been a resurgence in interest in Kosen judo in recent years due to its similarities with Brazilian jiu jitsu.
Brazilian Jiujitsu (BJJ) was introduced to Brazil through the Gracie family by judoka Mitsuyo Maeda. Maeda had significant experience fighting wrestlers and boxers; from this experience he developed a theory of combat that has evolved into modern MMA.
According to Renzo Gracie's book Mastering Jujitsu, Maeda not only taught the art of judo to Carlos Gracie, but also taught a particular philosophy about the nature of combat based on his travels competing and training alongside catch-wrestlers, boxers, savate fighters, and various other martial artists. The book details Maeda's theory that physical combat could be broken down into distinct phases, such as the striking phase, the grappling phase, the ground phase, and so on. Thus, it was a smart fighter's task to keep the fight located in the phase of combat that best suited his own strengths. The book further states that this theory was a fundamental influence on the Gracie approach to combat.

Wednesday, February 21, 2018

Postdoc in Theoretical Physics and Machine Learning

I am searching for a new postdoc. Please refer applicants to this MSU HR posting.
Postdoc in Theoretical Physics and Machine Learning

Stephen Hsu, Vice-President for Research and Professor of Physics at Michigan State University, anticipates filling a Research Associate (postdoctoral) position to start in the summer or fall of 2018. The successful applicant will have broad interests in theoretical physics and good computational skills. In addition to research in particle physics and cosmology, he or she will work on problems in machine learning and computational genomics.

Ongoing MSU theoretical physics research includes QCD theory and phenomenology, electroweak symmetry breaking mechanisms, supersymmetry and other beyond-the-standard-model scenarios, cosmology, and collider phenomenology. Recently, a new group of 3 theorists have been hired in the area of lattice QCD.

The Physics/Astronomy Department at MSU has 60 faculty members; it has strong research programs in Condensed Matter Physics, Nuclear Physics, and Astronomy, in addition to High Energy Physics (http://www.pa.msu.edu/hep/hept.html).


Sunday, February 18, 2018

Giorgio Moroder is Boss

See also The History of Synth Pop.
Wikipedia: In 1977 ... he co-wrote and produced the seminal Donna Summer hit single "I Feel Love",[5][8] the first track in the Hi-NRG genre. The following year he released "Chase", the theme from the film Midnight Express. ... A double album of the Foxes soundtrack was released on the disco label Casablanca Records which includes Donna Summer's hit single "On the Radio", which Moroder both produced and co-wrote. ... The American Gigolo soundtrack featured the Moroder-produced "Call Me" by Blondie, a US and UK number one hit. The combined club play of the album's tracks was number two for five weeks on the disco/dance charts.[9] In 1982 he wrote the soundtrack of the movie Cat People, including the hit single "Cat People (Putting Out Fire)" featuring David Bowie. In 1983, Moroder produced the soundtrack for the film Scarface. ... In 1986, Moroder collaborated with his protégé Harold Faltermeyer (of "Axel F") and lyricist Tom Whitlock to create the score for the film Top Gun (1986) which included Kenny Loggins' hit "Danger Zone" and Berlin's "Take My Breath Away".

0:00 Donna Summer - Hot Stuff 3:49 Donna Summer - I Feel Love 9:28 Giorgio Moroder - Chase 13:10 Donna Summer - Love to Love You Baby 16:35 Giorgio Moroder - From Here to Eternity 22:26 Blondie - Call Me 26:58 Japan - Life in Tokyo 29:29 Paul Engemann - Push it To the Limit 32:32 David Bowie - Cat People (Putting Out the Fire) 36:35 Giorgio Moroder - E=MC² 41:29 Amy Holland - She's On Fire (GTA version) 44:42 Irene Cara - What a Feeling 48:01 Giorgio Moroder - I'm Left, You're Right, She's Gone 53:03 Philip Oakey & Giorgio Moroder - Together in Electric Dreams 56:51 Berlin - Take My Breath Away 1:01:17 Elizabeth Daily - Shake it Up Tonight (GTA version) 1:04:24 Giorgio Moroder - I Wanna Rock You 1:10:52 Michael Sembello - Maniac 1:13:13 Donna Summer & Barbra Straisand - Enough is Enough 1:17:58 Daft Punk - Giorgio by Moroder

This track starts with a Moroder interview -- The sound of the future! :-)

Thursday, February 15, 2018

Genetic testing and embryo selection: current status and ethical issues

This is a conversation with two Stanford students about the current status of genetic testing of embryos in IVF, focusing on related ethical issues. Because there is a lot of interest in this topic I suggested we record the conversation and put it online.

I was at Stanford last fall to give a #nofilter talk on this subject, which is where I met one of the students in the video. When I was at Caltech we used to think of Stanford as kind of a soft place, where people didn't have to work so hard. Well, things are different now! My talk was on Friday evening in the Gates Computer Science Building. As I crossed campus I passed football fans (mostly alumni) clad in cardinal (Stanford) and purple (U Washington) heading toward the stadium. I was shocked to find the CS building full of students working really hard at 5:30pm on Friday! (Just like tech, back in the day, I thought.) I was told that 1000 students are enrolled in the machine learning course...

Frank Herbert interview on the origins of Dune (1969)

The interviewer is Willis E. McNelly, a professor of English (specializing in science fiction). Herbert discusses artistic as well as conceptual decisions made in the writing and background world building for Dune. Highly recommended for any fan of the book.

See also Dune and The Butlerian Jihad and Darwin Among the Machines.
The Bene Gesserit program had as its target the breeding of a person they labeled "Kwisatz Haderach," a term signifying "one who can be many places at once." In simpler terms, what they sought was a human with mental powers permitting him to understand and use higher order dimensions.

They were breeding for a super-Mentat, a human computer with some of the prescient abilities found in Guild navigators. Now, attend these facts carefully:

Muad'Dib, born Paul Atreides, was the son of the Duke Leto, a man whose bloodline had been watched carefully for more than a thousand years. The Prophet's mother, Lady Jessica, was a natural daughter of the Baron Vladimir Harkonnen and carried gene-markers whose supreme importance to the breeding program was known for almost two thousand years. She was a Bene Gesserit bred and trained, and should have been a willing tool of the project.

The Lady Jessica was ordered to produce an Atreides daughter. The plan was to inbreed this daughter with Feyd-Rautha Harkonnen, a nephew of the Baron Vladimir, with the high probability of a Kwisatz Haderach from that union. Instead, for reasons she confesses have never been completely clear to her, the concubine Lady Jessica defied her orders and bore a son. This alone should have alerted the Bene Gesserit to the possibility that a wild variable had entered their scheme. But there were other far more important indications that they virtually ignored ...
"Kwisatz Haderach" is similar to the Hebrew "Kefitzat Haderech", which literally means "contracting the path"; Herbert defines Kwisatz Haderach as "the Shortening of the Way" (Dune: Appendix IV).

Another good recording of Herbert, but much later in his life.

Sunday, February 11, 2018

The History of Synth Pop (video documentary)

I had a vague awareness of synth pop groups like Depeche Mode, Joy Division, New Order, Human League, OMD when I was growing up. I loved the music but knew almost nothing about the bands and the context from which they emerged. This documentary locates them in the post-punk, Kraftwerk-influenced UK of the late 1970s and early 1980s. Highly recommended if songs from these groups give you a jolt of exuberant nostalgia :-)

Now that I'm older I really enjoy this kind of exploration, in which writers, artists, scientists, entrepreneurs reminisce about their youthful moments of creation and discovery. How did it look at the time? And now, in the fullness of life? All those moments, lost in time like tears in rain.

I watched Atomic Blonde on a recent flight and, other than one long fight scene near the end -- "Stoy! Stoy!" (pleading... Bang!), found it mostly forgettable. But the incredible 1980s soundtrack got me thinking about this music again...

AI and Genomics, explained (2 videos)

This video is a nicely done short introduction to AI for non-specialists. It's part of Shift Change, a six part series on automation and the future of work.

I came across the video when creator Joss Fong (Vox) contacted me about her new project on human genomics and genomic prediction. As readers know I think the two most impactful technologies over the next 20-30 years will be AI and genomics. So Fong is on the right track...

This is the best (non-technical) video I've seen on the coming genomic revolution. However, it's from 2016 and does not focus on the machine learning / bioinformatic challenge of figuring out exactly which edits one should make -- i.e., the part of the problem I work on :-)  For the real thing, see here.

From a recent talk I gave at a biomedical research institute:
Your children and grandchildren will not just be competing against other people. They will also compete in the marketplace with machines. The code run on these machines is improving every day, thanks to AI. Will the DNA code run by your descendants also need to improve?


Of course it's not just about competition. How can we put a value on a healthy, long life? The ability to swim effortlessly across a pool or run fast and leap higher? To actually understand what Einstein did, in place of some vague second-hand words?

Friday, February 09, 2018

UFC 221: Rockhold vs Romero

Two superb athletes will meet at UFC 221 for the 185lb championship. I'd say 65% chance Rockhold wins, but I won't be shocked if Yoel explodes and KOs Rockhold with little warning.

Like Chael Sonnen (below) I am really excited to see them grapple -- one of the top MMA BJJ talents (Rockhold) versus a former World Champion in freestyle wrestling. Rockhold's top game is very strong -- he might be the first person ever to control and finish Romero.

See also Yoel Romero, freak athlete. He's 40 years old!

Great analysis of the fight from Firas Zahabi (Georges St. Pierre's coach) and Chael Sonnen:

Wednesday, February 07, 2018

US Needs a National AI Strategy: A Sputnik Moment?

The US needs a national AI strategy. Many academic researchers that could contribute to AI research -- including to fundamental new ideas and algorithms, mathematical frameworks for better understanding why some algorithms and architectures work better than others, etc. -- are not able to get involved at the real frontier because they lack the kind of curated data sets and large compute platforms that researchers at Google Brain or DeepMind have access to. Those resources are expensive, but necessary for rapid progress. We need national infrastructure platforms -- similar to physics user facilities like an accelerator or light source or telescope -- in order to support researchers at our universities and national labs doing work in machine learning, AI, and data science.

In contrast, China has articulated a very ambitious national AI plan which has them taking the lead sometime in the 2020s.

Eric Schmidt discusses these points in the video, declaring this a Sputnik moment:

Sunday, February 04, 2018

Steve Pinker and Joe Rogan

I've just started watching this so I can't give you an evaluation of the whole conversation. Looks promising -- they jump right in on topics like sex differences, political correctness, internet flame wars, the Trump candidacy, social media, ... (I'm skipping the Super Bowl, by the way. I stopped watching the NFL and NBA years ago.)

Pinker: "Virtue Signaling Fanatics are a Thing"  (at about 30min)

In case 2 hours of Steve Pinker is not enough for you, here's a panel he and I were on at the 92nd Street Y.

Thursday, February 01, 2018

Counting branches of the black hole wave function

When I was at Caltech a few weeks ago I had a chance to discuss the recent paper below by Sean Carroll and collaborators. (Authors are at Caltech, Berkeley, and UBC.)

Their paper is very clearly written, but probably suitable only for experts who are already familiar with the black hole information paradox. I discussed related ideas in 2013 papers Macroscopic superpositions and black hole unitarity and Factorization of unitarity and black hole firewalls.
Branches of the Black Hole Wave Function Need Not Contain Firewalls

Abstract: We discuss the branching structure of the quantum-gravitational wave function that describes the evaporation of a black hole. A global wave function which initially describes a classical Schwarzschild geometry is continually decohered into distinct semiclassical branches by the emission of Hawking radiation. The laws of quantum mechanics dictate that the wave function evolves unitarily, but this unitary evolution is only manifest when considering the global description of the wave function: it is not implemented by time evolution on a single semiclassical branch. Conversely, geometric notions like the position or smoothness of a horizon only make sense on the level of individual branches. We consider the implications of this picture for probes of black holes by classical observers in definite geometries, like those involved in the AMPS construction. We argue that individual branches can describe semiclassical geometries free of firewalls, even as the global wave function evolves unitarily. We show that the pointer states of infalling detectors that are robust under Hamiltonian evolution are distinct from, and incompatible with, those of exterior detectors stationary with respect to the black hole horizon, in the sense that the pointer bases are related to each other via nontrivial transformations that mix system, apparatus, and environment. This result describes a Hilbert-space version of black hole complementarity.
At question is a possible loophole in the AMPS argument that black hole (BH) firewalls are a necessary consequence of the assumption of unitary evolution (i.e., that the quantum information associated with things that fall into the hole eventually re-emerges in the Hawking radiation).

I pointed out that as a BH evaporates, fluctuations in the specific pattern of Hawking radiation lead to macroscopically different trajectories of the BH itself. The BH wave function is therefore a superposition of many branches which describe different spacetime geometries. The firewall construction, and many of the older arguments indicating a BH information paradox, assume a fixed semiclassical geometry. I noted that unitarity might be violated on each decoherent branch of the BH wave function, but restored when all the superpositions are added together.

Most of my discussions with AMPS, and their criticism of my papers, focused on counting the number of decoherent branches. They claimed it was obvious that there were not enough decoherent branches to restore unitarity, whereas I claimed that it was obvious that the number of decoherent branches was of the same order as the total number of Hawking radiation states. To construct the BH information paradox one has to assume a universe much larger than the BH, with many more degrees of freedom, so that each Hawking quantum that leaves the BH is decohered via interactions with the environment before reaching future infinity. Thus (I argued***) there are enough branches to (in principle) unitarize the separately non-unitary processes on each branch. Whether and how this actually occurs is still an open question! But it seems possible to me that the complex superposition structure of the "wave function of the universe" containing a BH plays a role in the information paradox.

Sean and his co-authors emphasize that AMPS have the burden of proof to show that summing over branches cannot unitarize the BH amplitude.

For more, see these posts:

Black hole firewalls and all that
Big brains battle black hole firewalls
Fuzzballs, black holes and firewalls

*** The difficult question is whether one should run the AMPS construction over a description that is coarse grained over many decoherent branches, thereby reducing significantly the effective total number.

Saturday, January 27, 2018

Mathematical Theory of Deep Neural Networks (Princeton workshop)

This looks interesting. Deep Learning would benefit from a stronger theoretical understanding of why it works so well. I hope they put the talks online!
Mathematical Theory of Deep Neural Networks

Tuesday March 20th, Princeton Neuroscience Institute.
PNI Psychology Lecture Hall 101

Recent advances in deep networks, combined with open, easily-accessible implementations, have moved empirical results far faster than formal understanding. The lack of rigorous analysis for these techniques limits their use in addressing scientific questions in the physical and biological sciences, and prevents systematic design of the next generation of networks. Recently, long-past-due theoretical results have begun to emerge. These results, and those that will follow in their wake, will begin to shed light on the properties of large, adaptive, distributed learning architectures, and stand to revolutionize how computer science and neuroscience understand these systems.

This intensive one-day technical workshop will focus on state of the art theoretical understanding of deep learning. We aim to bring together researchers from the Princeton Neuroscience Institute (PNI) and of the theoretical machine learning group at the Institute for Advanced Studies (IAS) interested in more rigorously understanding deep networks to foster increased discussion and collaboration across these intrinsically related groups.

Wednesday, January 24, 2018

The Content of their Character: Ed Blum and Jian Li

See 20 years @15 percent: does Harvard discriminate against Asian-Americans? The excerpt below is from the Harvard lawsuit brief, recalling the parallel between what had been done to limit Jewish enrollment in the early 20th century, and the current situation with Asian-Americans.
... Harvard is engaging in racial balancing. Over an extended period, Harvard’s admission and enrollment figures for each racial category have shown almost no change. Each year, Harvard admits and enrolls essentially the same percentage of African Americans, Hispanics, whites, and Asian Americans even though the application rates and qualifications for each racial group have undergone significant changes over time. This is not the coincidental byproduct of an admissions system that treats each applicant as an individual; indeed, the statistical evidence shows that Harvard modulates its racial admissions preference whenever there is an unanticipated change in the yield rate of a particular racial group in the prior year. Harvard’s remarkably stable admissions and enrollment figures over time are the deliberate result of systemwide intentional racial discrimination designed to achieve a predetermined racial balance of its student body.

... In a letter to the chairman of the committee, President Lowell wrote that “questions of race,” though “delicate and disagreeable,” were not solved by ignoring them. The solution was a new admissions system giving the school wide discretion to limit the admission of Jewish applicants: “To prevent a dangerous increase in the proportion of Jews, I know at present only one way which is at the same time straightforward and effective, and that is a selection by a personal estimate of character on the part of the Admissions authorities ... The only way to make a selection is to limit the numbers, accepting those who appear to be the best.”

... The reduction in Jewish enrollment at Harvard was immediate. The Jewish portion of Harvard’s entering class dropped from over 27 percent in 1925 to 15 percent the following year. For the next 20 years, this percentage (15 percent) remained virtually unchanged.

... The new policy permitted the rejection of scholastically brilliant students considered “undesirable,” and it granted the director of admissions broad latitude to admit those of good background with weaker academic records. The key code word used was “character” — a quality thought to be frequently lacking among Jewish applicants, but present congenitally among affluent Protestants.
DOJ invokes Title VI against Harvard admissions:
WSJ: ... The Justice Department, whose Civil Rights Division is conducting the investigation into similar allegations, said in a letter to Harvard’s lawyers, dated Nov. 17 and reviewed by the Journal, that the school was being investigated under Title VI of the Civil Rights Act of 1964, which bars discrimination on the basis of race, color and national origin for organizations that receive federal funding. The letter also said the school had failed to comply with a Nov. 2 deadline to provide documents related to the university’s admissions policies and practices. ...
I believe I first mentioned Jian Li on this blog back in 2006! It's nice to see that he is still courageous and principled today.

From his closing remarks:
I have a message to every single Asian-American student in the country who is applying to college: your civil rights are being violated and you must speak up in defense of them. If you've suffered discrimination you have the option to file a complaint with the Office for Civil Rights. Let your voice be heard .. not only through formal means but also by simply letting it be known in your schools and your communities, in the press and on social media, that university discrimination is pervasive and that this does not sit well with you. Together we will fight to ensure that universities can no longer treat us as second-class citizens.

Friday, January 19, 2018

Allen Institute meeting on Genetics of Complex Traits

You can probably tell by all the photos below that I love their new building :-)

I was a participant in this event: What Makes Us Human? The Genetics of Complex Traits (Allen Frontiers Group), including in a small second day workshop with just the speakers and the AI leadership. This workshop will, I hope, result in some interesting new initiatives in complex trait genomics!

I'd like to thank the Allen Institute organizers for making this such a pleasant and productive 2 days. I learned some incredible things from the other speakers and I recommend all of their talks -- available here.

My talk:

Action photos:

Working hard on day 2 in the little conference room :-)

Tuesday, January 16, 2018

The Jiujitsu Philosopher: John Danaher

John Danaher is one of the deepest thinkers in combat sports, MMA, and jiujitsu. He has coached a number of world champions in MMA and jiujitsu/submission grappling (Georges St. Pierre, Garry Tonon, etc.). The recent leg lock technique renaissance is largely due to Danaher and his school.

Danaher was a philosophy PhD student at Columbia before discovering BJJ through Renzo Gracie's academy in NYC. When I was a Yale professor (in the 90s) I made trips to Renzo's for training. I don't recall Danaher (who would have been a student/instructor there at the time), but I do recall Craig Kukuk, Renzo's partner in the school and the first US blackbelt instructor. Kukuk had played linebacker at Iowa State University (where I grew up), and we spent time talking about Iowa (a big wrestling hotbed) and the origins of jiujitsu and ultimate fighting in the US. I had trained in Japan and so knew quite a bit about the relationship between traditional judo and BJJ. At one time I probably knew as much as anyone about the relationship between judo, BJJ, MMA, and US folk style wrestling.

See Mama said knock you out.

What Makes Us Human? The Genetics of Complex Traits (Allen Frontiers Group)

I'll be attending this meeting in Seattle the next few days.
Recent research has led to new insights on how genes shape brain structure and development, and their impact on individual variation. Although significant inroads have been made in understanding the genetics underlying disease risk, what about the complex traits of extraordinary variation - such as cognition, superior memory, etc.? Can current advances shed light on genetic components underpinning these variations?

Personal genomics, biobank resources, emerging statistical genetics methods and neuroimaging capabilities are opening new frontiers in the field of complex trait analysis. This symposium will highlight experts using diverse approaches to explore a spectrum of individual variation of the human mind.
Paul Allen (MSFT co-founder) is a major supporter of scientific research, including the Allen Institute for Brain Science. Excerpts from his memoir, Idea Man.
We are at a unique moment in bioscience. New ideas, combined with emerging technologies, will create unprecedented and transformational insights into living systems. Accelerating the pace of this change requires a thoughtful and agile exploration of the entire landscape of bioscience, across disciplines and spheres of research. Launched in 2016 with a $100 million commitment toward a larger 10-year plan, The Paul G. Allen Frontiers Group will discover and support scientific ideas that change the world. We are committed to a continuous conversation with the scientific community that allows us to remain at the ever-changing frontiers of science and reimagine what is possible.
My talk is scheduled for 3:55 PM Pacific Weds 1/17. All talks will be streamed on the Allen Institute Facebook page.

Saturday, January 06, 2018

Institute for Advanced Study: Genomic Prediction of Complex Traits (seminar)

Genomic Prediction of Complex Traits

After a brief review (suitable for physicists) of computational genomics and complex traits, I describe recent progress in this area. Using methods from Compressed Sensing (L1-penalized regression; Donoho-Tanner phase transition with noise) and the UK BioBank dataset of 500k SNP genotypes, we construct genomic predictors for several complex traits. Our height predictor captures nearly all of the predicted SNP heritability for this trait -- thereby resolving the missing heritability problem. Actual heights of most individuals in validation tests are within a few cm of predicted heights. I also discuss application of these methods to cognitive ability and polygenic disease risk: sparsity estimates (of the number of causal loci), combined with phase transition scaling analysis, allow estimates of the amount of data required to construct good predictors. Finally, I discuss how these advances will affect human health and reproduction (embryo selection for In Vitro Fertilization, genetic editing) in the coming decade.

Steve Hsu

Michigan State University

I recently gave a similar talk at 23andMe (slides at link).

Note Added: Many people asked for video of this talk, but alas recording talks is not standard practice at IAS. I did give a similar talk using the same slides just a week later at the Allen Institute in Seattle (Symposium on Genetics of Complex Traits): video here.

Some Comments and Slides:

I tried to make the talk understandable to physicists, and at least according to what I was told (and my impression from the questions asked during and after the talk), largely succeeded. Early on, when presenting the phenotype function y(g), both Nima Arkani-Hamed (my host) and Ed Witten asked some questions about the "units" of the various quantities involved. In the actual computation everything is z-scored: measured in units of SD relative to the sample mean. I didn't realize until later that there was some confusion about how this is done for the "state variable" of the genetic locus g_i. In fact, when the gene array is read the result is 0,1,2 for homozygous common allele, heterozygous, homozygous rare allele, respectively. (I might have that backwards but you get the point.) For each locus there is a minor allele frequency (MAF) and this determines the sample average and SD of the distribution of 0's, 1's, and 2's. It is the z-scored version of this variable that appears in the computation. I didn't realize certain people were following the details so closely in the talk but I should not be surprised ;-) In the future I'll include a slide specifically on this to avoid confusion.

Looking at my slide on missing heritability, Witten immediately noted that estimating SNP heritability (as opposed to total or broad sense heritability) is nontrivial and I had to quickly explain the GCTA technique!

During the talk I discussed the theoretical reason we expect to find a lot of additive variance: nonlinear gadgets are fragile (easy to break through recombination in sexual reproduction), whereas additive genetic variance can be reliably passed on and is easy for natural selection to act on***. (See also Fisher's Fundamental Theorem of Natural Selection. More.) Usually these comments pass over the head of the audience but at IAS I am sure quite a few people understood the point.

One non-physicist reader of this blog braved IAS security and managed to attend the lecture. I am flattered, and I invite him to share his impressions in the comments!

Afterwards there was quite a bit of additional discussion which spilled over into tea time. The important ideas: how Compressed Sensing works, the nature of the phase transition, how we can predict the amount of data required to build a good predictor (capturing most of the SNP heritability) using the universality of the phase transition + estimate of sparsity, etc. were clearly absorbed by the people I talked to.


*** On the genetic architecture of intelligence and other quantitative traits (p.16):
... The preceding discussion is not intended to convey an overly simplistic view of genetics or systems biology. Complex nonlinear genetic systems certainly exist and are realized in every organism. However, quantitative differences between individuals within a species may be largely due to independent linear effects of specific genetic variants. As noted, linear effects are the most readily evolvable in response to selection, whereas nonlinear gadgets are more likely to be fragile to small changes. (Evolutionary adaptations requiring significant changes to nonlinear gadgets are improbable and therefore require exponentially more time than simple adjustment of frequencies of alleles of linear effect.) One might say that, to first approximation, Biology = linear combinations of nonlinear gadgets, and most of the variation between individuals is in the (linear) way gadgets are combined, rather than in the realization of different gadgets in different individuals.

Linear models work well in practice, allowing, for example, SNP-based prediction of quantitative traits (milk yield, fat and protein content, productive life, etc.) in dairy cattle. ...

Friday, January 05, 2018

Gork revisited, 2018

It's been almost 10 years since I made the post Are you Gork?

Over the last decade, both scientists and non-scientists have become more confident that we will someday create:

A. AGI (= sentient AI, named "Gork" :-)  See Rise of the Machines: Survey of AI Researchers.

B. Quantum Computers. See Quantum Computing at a Tipping Point?

This change in zeitgeist makes the thought experiment proposed below much less outlandish. What, exactly, does Gork perceive? Why couldn't you be Gork? (Note that the AGI in Gork can be an entirely classical algorithm even though he exists in a quantum simulation.)

Slide from this [Caltech IQI] talk. See also illustrations in Big Ed.
Survey questions:

1) Could you be Gork the robot? (Do you split into different branches after observing the outcome of, e.g., a Stern-Gerlach measurement?)

2) If not, why? e.g.,

I have a soul and Gork doesn't!  Copenhagen people, please use exit on your left.

Decoherence solved all that!  Sorry, try again. See previous post.

I don't believe that quantum computers will work as designed, e.g., sufficiently large algorithms or subsystems will lead to real (truly irreversible) collapse. Macroscopic superpositions that are too big (larger than whatever was done in the lab last week!) are impossible.

QM is only an algorithm for computing probabilities -- there is no reality to the quantum state or wavefunction or description of what is happening inside a quantum computer. Tell this to Gork!

Stop bothering me -- I only care about real stuff like the Higgs mass / SUSY-breaking scale / string Landscape / mechanism for high-Tc / LIBOR spread / how to generate alpha. 
[ 2018: Ha Ha -- first 3 real stuff topics turned out to be pretty boring use of the last decade... ]
Just as A. and B. above have become less outlandish assumptions, our ability to create large and complex superposition states with improved technology (largely developed for quantum computing; see Schrodinger's Virus) will make the possibility that we ourselves exist in a superposition state less shocking. Future generations of physicists will wonder why it took their predecessors so long to accept Many Worlds.

Bonus! I will be visiting Caltech next week (Tues and Weds 1/8-9). Any blog readers interested in getting a coffee or beer please feel free to contact me :-)

Blog Archive