Text

Physicist, Startup Founder, Blogger, Dad

Sunday, February 18, 2018

Giorgio Moroder is Boss


See also The History of Synth Pop.
Wikipedia: In 1977 ... he co-wrote and produced the seminal Donna Summer hit single "I Feel Love",[5][8] the first track in the Hi-NRG genre. The following year he released "Chase", the theme from the film Midnight Express. ... A double album of the Foxes soundtrack was released on the disco label Casablanca Records which includes Donna Summer's hit single "On the Radio", which Moroder both produced and co-wrote. ... The American Gigolo soundtrack featured the Moroder-produced "Call Me" by Blondie, a US and UK number one hit. The combined club play of the album's tracks was number two for five weeks on the disco/dance charts.[9] In 1982 he wrote the soundtrack of the movie Cat People, including the hit single "Cat People (Putting Out Fire)" featuring David Bowie. In 1983, Moroder produced the soundtrack for the film Scarface. ... In 1986, Moroder collaborated with his protégé Harold Faltermeyer (of "Axel F") and lyricist Tom Whitlock to create the score for the film Top Gun (1986) which included Kenny Loggins' hit "Danger Zone" and Berlin's "Take My Breath Away".



TRACK LISTING  
0:00 Donna Summer - Hot Stuff 3:49 Donna Summer - I Feel Love 9:28 Giorgio Moroder - Chase 13:10 Donna Summer - Love to Love You Baby 16:35 Giorgio Moroder - From Here to Eternity 22:26 Blondie - Call Me 26:58 Japan - Life in Tokyo 29:29 Paul Engemann - Push it To the Limit 32:32 David Bowie - Cat People (Putting Out the Fire) 36:35 Giorgio Moroder - E=MC² 41:29 Amy Holland - She's On Fire (GTA version) 44:42 Irene Cara - What a Feeling 48:01 Giorgio Moroder - I'm Left, You're Right, She's Gone 53:03 Philip Oakey & Giorgio Moroder - Together in Electric Dreams 56:51 Berlin - Take My Breath Away 1:01:17 Elizabeth Daily - Shake it Up Tonight (GTA version) 1:04:24 Giorgio Moroder - I Wanna Rock You 1:10:52 Michael Sembello - Maniac 1:13:13 Donna Summer & Barbra Straisand - Enough is Enough 1:17:58 Daft Punk - Giorgio by Moroder

Thursday, February 15, 2018

Genetic testing and embryo selection: current status and ethical issues



This is a conversation with two Stanford students about the current status of genetic testing of embryos in IVF, focusing on related ethical issues. Because there is a lot of interest in this topic I suggested we record the conversation and put it online.

I was at Stanford last fall to give a #nofilter talk on this subject, which is where I met one of the students in the video. When I was at Caltech we used to think of Stanford as kind of a soft place, where people didn't have to work so hard. Well, things are different now! My talk was on Friday evening in the Gates Computer Science Building. As I crossed campus I passed football fans (mostly alumni) clad in cardinal (Stanford) and purple (U Washington) heading toward the stadium. I was shocked to find the CS building full of students working really hard at 5:30pm on Friday! (Just like tech, back in the day, I thought.) I was told that 1000 students are enrolled in the machine learning course...



Frank Herbert interview on the origins of Dune (1969)



The interviewer is Willis E. McNelly, a professor of English (specializing in science fiction). Herbert discusses artistic as well as conceptual decisions made in the writing and background world building for Dune. Highly recommended for any fan of the book.

See also Dune and The Butlerian Jihad and Darwin Among the Machines.
The Bene Gesserit program had as its target the breeding of a person they labeled "Kwisatz Haderach," a term signifying "one who can be many places at once." In simpler terms, what they sought was a human with mental powers permitting him to understand and use higher order dimensions.

They were breeding for a super-Mentat, a human computer with some of the prescient abilities found in Guild navigators. Now, attend these facts carefully:

Muad'Dib, born Paul Atreides, was the son of the Duke Leto, a man whose bloodline had been watched carefully for more than a thousand years. The Prophet's mother, Lady Jessica, was a natural daughter of the Baron Vladimir Harkonnen and carried gene-markers whose supreme importance to the breeding program was known for almost two thousand years. She was a Bene Gesserit bred and trained, and should have been a willing tool of the project.

The Lady Jessica was ordered to produce an Atreides daughter. The plan was to inbreed this daughter with Feyd-Rautha Harkonnen, a nephew of the Baron Vladimir, with the high probability of a Kwisatz Haderach from that union. Instead, for reasons she confesses have never been completely clear to her, the concubine Lady Jessica defied her orders and bore a son. This alone should have alerted the Bene Gesserit to the possibility that a wild variable had entered their scheme. But there were other far more important indications that they virtually ignored ...
"Kwisatz Haderach" is similar to the Hebrew "Kefitzat Haderech", which literally means "contracting the path"; Herbert defines Kwisatz Haderach as "the Shortening of the Way" (Dune: Appendix IV).

Another good recording of Herbert, but much later in his life.

Sunday, February 11, 2018

The History of Synth Pop (video documentary)



I had a vague awareness of synth pop groups like Depeche Mode, Joy Division, New Order, Human League, OMD when I was growing up. I loved the music but knew almost nothing about the bands and the context from which they emerged. This documentary locates them in the post-punk, Kraftwerk-influenced UK of the late 1970s and early 1980s. Highly recommended if songs from these groups give you a jolt of exuberant nostalgia :-)

Now that I'm older I really enjoy this kind of exploration, in which writers, artists, scientists, entrepreneurs reminisce about their youthful moments of creation and discovery. How did it look at the time? And now, in the fullness of life? All those moments, lost in time like tears in rain.

I watched Atomic Blonde on a recent flight and, other than one long fight scene near the end -- "Stoy! Stoy!" (pleading... Bang!), found it mostly forgettable. But the incredible 1980s soundtrack got me thinking about this music again...

AI and Genomics, explained (2 videos)

This video is a nicely done short introduction to AI for non-specialists. It's part of Shift Change, a six part series on automation and the future of work.



I came across the video when creator Joss Fong (Vox) contacted me about her new project on human genomics and genomic prediction. As readers know I think the two most impactful technologies over the next 20-30 years will be AI and genomics. So Fong is on the right track...

This is the best (non-technical) video I've seen on the coming genomic revolution. However, it's from 2016 and does not focus on the machine learning / bioinformatic challenge of figuring out exactly which edits one should make -- i.e., the part of the problem I work on :-)  For the real thing, see here.



From a recent talk I gave at a biomedical research institute:
Your children and grandchildren will not just be competing against other people. They will also compete in the marketplace with machines. The code run on these machines is improving every day, thanks to AI. Will the DNA code run by your descendants also need to improve?

...

Of course it's not just about competition. How can we put a value on a healthy, long life? The ability to swim effortlessly across a pool or run fast and leap higher? To actually understand what Einstein did, in place of some vague second-hand words?

Friday, February 09, 2018

UFC 221: Rockhold vs Romero

Two superb athletes will meet at UFC 221 for the 185lb championship. I'd say 65% chance Rockhold wins, but I won't be shocked if Yoel explodes and KOs Rockhold with little warning.

Like Chael Sonnen (below) I am really excited to see them grapple -- one of the top MMA BJJ talents (Rockhold) versus a former World Champion in freestyle wrestling. Rockhold's top game is very strong -- he might be the first person ever to control and finish Romero.





See also Yoel Romero, freak athlete. He's 40 years old!

Great analysis of the fight from Firas Zahabi (Georges St. Pierre's coach) and Chael Sonnen:



Wednesday, February 07, 2018

US Needs a National AI Strategy: A Sputnik Moment?

The US needs a national AI strategy. Many academic researchers that could contribute to AI research -- including to fundamental new ideas and algorithms, mathematical frameworks for better understanding why some algorithms and architectures work better than others, etc. -- are not able to get involved at the real frontier because they lack the kind of curated data sets and large compute platforms that researchers at Google Brain or DeepMind have access to. Those resources are expensive, but necessary for rapid progress. We need national infrastructure platforms -- similar to physics user facilities like an accelerator or light source or telescope -- in order to support researchers at our universities and national labs doing work in machine learning, AI, and data science.

In contrast, China has articulated a very ambitious national AI plan which has them taking the lead sometime in the 2020s.

Eric Schmidt discusses these points in the video, declaring this a Sputnik moment:

Sunday, February 04, 2018

Steve Pinker and Joe Rogan



I've just started watching this so I can't give you an evaluation of the whole conversation. Looks promising -- they jump right in on topics like sex differences, political correctness, internet flame wars, the Trump candidacy, social media, ... (I'm skipping the Super Bowl, by the way. I stopped watching the NFL and NBA years ago.)

Pinker: "Virtue Signaling Fanatics are a Thing"  (at about 30min)

In case 2 hours of Steve Pinker is not enough for you, here's a panel he and I were on at the 92nd Street Y.

Thursday, February 01, 2018

Counting branches of the black hole wave function



When I was at Caltech a few weeks ago I had a chance to discuss the recent paper below by Sean Carroll and collaborators. (Authors are at Caltech, Berkeley, and UBC.)

Their paper is very clearly written, but probably suitable only for experts who are already familiar with the black hole information paradox. I discussed related ideas in 2013 papers Macroscopic superpositions and black hole unitarity and Factorization of unitarity and black hole firewalls.
Branches of the Black Hole Wave Function Need Not Contain Firewalls
https://arxiv.org/pdf/1712.04955.pdf

Abstract: We discuss the branching structure of the quantum-gravitational wave function that describes the evaporation of a black hole. A global wave function which initially describes a classical Schwarzschild geometry is continually decohered into distinct semiclassical branches by the emission of Hawking radiation. The laws of quantum mechanics dictate that the wave function evolves unitarily, but this unitary evolution is only manifest when considering the global description of the wave function: it is not implemented by time evolution on a single semiclassical branch. Conversely, geometric notions like the position or smoothness of a horizon only make sense on the level of individual branches. We consider the implications of this picture for probes of black holes by classical observers in definite geometries, like those involved in the AMPS construction. We argue that individual branches can describe semiclassical geometries free of firewalls, even as the global wave function evolves unitarily. We show that the pointer states of infalling detectors that are robust under Hamiltonian evolution are distinct from, and incompatible with, those of exterior detectors stationary with respect to the black hole horizon, in the sense that the pointer bases are related to each other via nontrivial transformations that mix system, apparatus, and environment. This result describes a Hilbert-space version of black hole complementarity.
At question is a possible loophole in the AMPS argument that black hole (BH) firewalls are a necessary consequence of the assumption of unitary evolution (i.e., that the quantum information associated with things that fall into the hole eventually re-emerges in the Hawking radiation).

I pointed out that as a BH evaporates, fluctuations in the specific pattern of Hawking radiation lead to macroscopically different trajectories of the BH itself. The BH wave function is therefore a superposition of many branches which describe different spacetime geometries. The firewall construction, and many of the older arguments indicating a BH information paradox, assume a fixed semiclassical geometry. I noted that unitarity might be violated on each decoherent branch of the BH wave function, but restored when all the superpositions are added together.

Most of my discussions with AMPS, and their criticism of my papers, focused on counting the number of decoherent branches. They claimed it was obvious that there were not enough decoherent branches to restore unitarity, whereas I claimed that it was obvious that the number of decoherent branches was of the same order as the total number of Hawking radiation states. To construct the BH information paradox one has to assume a universe much larger than the BH, with many more degrees of freedom, so that each Hawking quantum that leaves the BH is decohered via interactions with the environment before reaching future infinity. Thus (I argued***) there are enough branches to (in principle) unitarize the separately non-unitary processes on each branch. Whether and how this actually occurs is still an open question! But it seems possible to me that the complex superposition structure of the "wave function of the universe" containing a BH plays a role in the information paradox.

Sean and his co-authors emphasize that AMPS have the burden of proof to show that summing over branches cannot unitarize the BH amplitude.

For more, see these posts:

Black hole firewalls and all that
Big brains battle black hole firewalls
Fuzzballs, black holes and firewalls


*** The difficult question is whether one should run the AMPS construction over a description that is coarse grained over many decoherent branches, thereby reducing significantly the effective total number.

Saturday, January 27, 2018

Mathematical Theory of Deep Neural Networks (Princeton workshop)

This looks interesting. Deep Learning would benefit from a stronger theoretical understanding of why it works so well. I hope they put the talks online!
Mathematical Theory of Deep Neural Networks

Tuesday March 20th, Princeton Neuroscience Institute.
PNI Psychology Lecture Hall 101

Recent advances in deep networks, combined with open, easily-accessible implementations, have moved empirical results far faster than formal understanding. The lack of rigorous analysis for these techniques limits their use in addressing scientific questions in the physical and biological sciences, and prevents systematic design of the next generation of networks. Recently, long-past-due theoretical results have begun to emerge. These results, and those that will follow in their wake, will begin to shed light on the properties of large, adaptive, distributed learning architectures, and stand to revolutionize how computer science and neuroscience understand these systems.

This intensive one-day technical workshop will focus on state of the art theoretical understanding of deep learning. We aim to bring together researchers from the Princeton Neuroscience Institute (PNI) and of the theoretical machine learning group at the Institute for Advanced Studies (IAS) interested in more rigorously understanding deep networks to foster increased discussion and collaboration across these intrinsically related groups.

Wednesday, January 24, 2018

The Content of their Character: Ed Blum and Jian Li



See 20 years @15 percent: does Harvard discriminate against Asian-Americans? The excerpt below is from the Harvard lawsuit brief, recalling the parallel between what had been done to limit Jewish enrollment in the early 20th century, and the current situation with Asian-Americans.
... Harvard is engaging in racial balancing. Over an extended period, Harvard’s admission and enrollment figures for each racial category have shown almost no change. Each year, Harvard admits and enrolls essentially the same percentage of African Americans, Hispanics, whites, and Asian Americans even though the application rates and qualifications for each racial group have undergone significant changes over time. This is not the coincidental byproduct of an admissions system that treats each applicant as an individual; indeed, the statistical evidence shows that Harvard modulates its racial admissions preference whenever there is an unanticipated change in the yield rate of a particular racial group in the prior year. Harvard’s remarkably stable admissions and enrollment figures over time are the deliberate result of systemwide intentional racial discrimination designed to achieve a predetermined racial balance of its student body.

... In a letter to the chairman of the committee, President Lowell wrote that “questions of race,” though “delicate and disagreeable,” were not solved by ignoring them. The solution was a new admissions system giving the school wide discretion to limit the admission of Jewish applicants: “To prevent a dangerous increase in the proportion of Jews, I know at present only one way which is at the same time straightforward and effective, and that is a selection by a personal estimate of character on the part of the Admissions authorities ... The only way to make a selection is to limit the numbers, accepting those who appear to be the best.”

... The reduction in Jewish enrollment at Harvard was immediate. The Jewish portion of Harvard’s entering class dropped from over 27 percent in 1925 to 15 percent the following year. For the next 20 years, this percentage (15 percent) remained virtually unchanged.

... The new policy permitted the rejection of scholastically brilliant students considered “undesirable,” and it granted the director of admissions broad latitude to admit those of good background with weaker academic records. The key code word used was “character” — a quality thought to be frequently lacking among Jewish applicants, but present congenitally among affluent Protestants.
DOJ invokes Title VI against Harvard admissions:
WSJ: ... The Justice Department, whose Civil Rights Division is conducting the investigation into similar allegations, said in a letter to Harvard’s lawyers, dated Nov. 17 and reviewed by the Journal, that the school was being investigated under Title VI of the Civil Rights Act of 1964, which bars discrimination on the basis of race, color and national origin for organizations that receive federal funding. The letter also said the school had failed to comply with a Nov. 2 deadline to provide documents related to the university’s admissions policies and practices. ...
I believe I first mentioned Jian Li on this blog back in 2006! It's nice to see that he is still courageous and principled today.



From his closing remarks:
I have a message to every single Asian-American student in the country who is applying to college: your civil rights are being violated and you must speak up in defense of them. If you've suffered discrimination you have the option to file a complaint with the Office for Civil Rights. Let your voice be heard .. not only through formal means but also by simply letting it be known in your schools and your communities, in the press and on social media, that university discrimination is pervasive and that this does not sit well with you. Together we will fight to ensure that universities can no longer treat us as second-class citizens.

Friday, January 19, 2018

Allen Institute meeting on Genetics of Complex Traits

You can probably tell by all the photos below that I love their new building :-)

I was a participant in this event: What Makes Us Human? The Genetics of Complex Traits (Allen Frontiers Group), including in a small second day workshop with just the speakers and the AI leadership. This workshop will, I hope, result in some interesting new initiatives in complex trait genomics!

I'd like to thank the Allen Institute organizers for making this such a pleasant and productive 2 days. I learned some incredible things from the other speakers and I recommend all of their talks -- available here.

My talk:




Action photos:








Working hard on day 2 in the little conference room :-)

Tuesday, January 16, 2018

The Jiujitsu Philosopher: John Danaher



John Danaher is one of the deepest thinkers in combat sports, MMA, and jiujitsu. He has coached a number of world champions in MMA and jiujitsu/submission grappling (Georges St. Pierre, Garry Tonon, etc.). The recent leg lock technique renaissance is largely due to Danaher and his school.

Danaher was a philosophy PhD student at Columbia before discovering BJJ through Renzo Gracie's academy in NYC. When I was a Yale professor (in the 90s) I made trips to Renzo's for training. I don't recall Danaher (who would have been a student/instructor there at the time), but I do recall Craig Kukuk, Renzo's partner in the school and the first US blackbelt instructor. Kukuk had played linebacker at Iowa State University (where I grew up), and we spent time talking about Iowa (a big wrestling hotbed) and the origins of jiujitsu and ultimate fighting in the US. I had trained in Japan and so knew quite a bit about the relationship between traditional judo and BJJ. At one time I probably knew as much as anyone about the relationship between judo, BJJ, MMA, and US folk style wrestling.

See Mama said knock you out.

What Makes Us Human? The Genetics of Complex Traits (Allen Frontiers Group)


I'll be attending this meeting in Seattle the next few days.
Recent research has led to new insights on how genes shape brain structure and development, and their impact on individual variation. Although significant inroads have been made in understanding the genetics underlying disease risk, what about the complex traits of extraordinary variation - such as cognition, superior memory, etc.? Can current advances shed light on genetic components underpinning these variations?

Personal genomics, biobank resources, emerging statistical genetics methods and neuroimaging capabilities are opening new frontiers in the field of complex trait analysis. This symposium will highlight experts using diverse approaches to explore a spectrum of individual variation of the human mind.
Paul Allen (MSFT co-founder) is a major supporter of scientific research, including the Allen Institute for Brain Science. Excerpts from his memoir, Idea Man.
We are at a unique moment in bioscience. New ideas, combined with emerging technologies, will create unprecedented and transformational insights into living systems. Accelerating the pace of this change requires a thoughtful and agile exploration of the entire landscape of bioscience, across disciplines and spheres of research. Launched in 2016 with a $100 million commitment toward a larger 10-year plan, The Paul G. Allen Frontiers Group will discover and support scientific ideas that change the world. We are committed to a continuous conversation with the scientific community that allows us to remain at the ever-changing frontiers of science and reimagine what is possible.
My talk is scheduled for 3:55 PM Pacific Weds 1/17. All talks will be streamed on the Allen Institute Facebook page.

Saturday, January 06, 2018

Institute for Advanced Study: Genomic Prediction of Complex Traits (seminar)


Genomic Prediction of Complex Traits

After a brief review (suitable for physicists) of computational genomics and complex traits, I describe recent progress in this area. Using methods from Compressed Sensing (L1-penalized regression; Donoho-Tanner phase transition with noise) and the UK BioBank dataset of 500k SNP genotypes, we construct genomic predictors for several complex traits. Our height predictor captures nearly all of the predicted SNP heritability for this trait -- thereby resolving the missing heritability problem. Actual heights of most individuals in validation tests are within a few cm of predicted heights. I also discuss application of these methods to cognitive ability and polygenic disease risk: sparsity estimates (of the number of causal loci), combined with phase transition scaling analysis, allow estimates of the amount of data required to construct good predictors. Finally, I discuss how these advances will affect human health and reproduction (embryo selection for In Vitro Fertilization, genetic editing) in the coming decade.

FEATURING
Steve Hsu

SPEAKER AFFILIATION
Michigan State University

I recently gave a similar talk at 23andMe (slides at link).

Note Added: Many people asked for video of this talk, but alas recording talks is not standard practice at IAS. I did give a similar talk using the same slides just a week later at the Allen Institute in Seattle (Symposium on Genetics of Complex Traits): video here.


Some Comments and Slides:

I tried to make the talk understandable to physicists, and at least according to what I was told (and my impression from the questions asked during and after the talk), largely succeeded. Early on, when presenting the phenotype function y(g), both Nima Arkani-Hamed (my host) and Ed Witten asked some questions about the "units" of the various quantities involved. In the actual computation everything is z-scored: measured in units of SD relative to the sample mean. I didn't realize until later that there was some confusion about how this is done for the "state variable" of the genetic locus g_i. In fact, when the gene array is read the result is 0,1,2 for homozygous common allele, heterozygous, homozygous rare allele, respectively. (I might have that backwards but you get the point.) For each locus there is a minor allele frequency (MAF) and this determines the sample average and SD of the distribution of 0's, 1's, and 2's. It is the z-scored version of this variable that appears in the computation. I didn't realize certain people were following the details so closely in the talk but I should not be surprised ;-) In the future I'll include a slide specifically on this to avoid confusion.

Looking at my slide on missing heritability, Witten immediately noted that estimating SNP heritability (as opposed to total or broad sense heritability) is nontrivial and I had to quickly explain the GCTA technique!

During the talk I discussed the theoretical reason we expect to find a lot of additive variance: nonlinear gadgets are fragile (easy to break through recombination in sexual reproduction), whereas additive genetic variance can be reliably passed on and is easy for natural selection to act on***. (See also Fisher's Fundamental Theorem of Natural Selection. More.) Usually these comments pass over the head of the audience but at IAS I am sure quite a few people understood the point.

One non-physicist reader of this blog braved IAS security and managed to attend the lecture. I am flattered, and I invite him to share his impressions in the comments!

Afterwards there was quite a bit of additional discussion which spilled over into tea time. The important ideas: how Compressed Sensing works, the nature of the phase transition, how we can predict the amount of data required to build a good predictor (capturing most of the SNP heritability) using the universality of the phase transition + estimate of sparsity, etc. were clearly absorbed by the people I talked to.

Slides


*** On the genetic architecture of intelligence and other quantitative traits (p.16):
... The preceding discussion is not intended to convey an overly simplistic view of genetics or systems biology. Complex nonlinear genetic systems certainly exist and are realized in every organism. However, quantitative differences between individuals within a species may be largely due to independent linear effects of specific genetic variants. As noted, linear effects are the most readily evolvable in response to selection, whereas nonlinear gadgets are more likely to be fragile to small changes. (Evolutionary adaptations requiring significant changes to nonlinear gadgets are improbable and therefore require exponentially more time than simple adjustment of frequencies of alleles of linear effect.) One might say that, to first approximation, Biology = linear combinations of nonlinear gadgets, and most of the variation between individuals is in the (linear) way gadgets are combined, rather than in the realization of different gadgets in different individuals.

Linear models work well in practice, allowing, for example, SNP-based prediction of quantitative traits (milk yield, fat and protein content, productive life, etc.) in dairy cattle. ...

Friday, January 05, 2018

Gork revisited, 2018

It's been almost 10 years since I made the post Are you Gork?

Over the last decade, both scientists and non-scientists have become more confident that we will someday create:

A. AGI (= sentient AI, named "Gork" :-)  See Rise of the Machines: Survey of AI Researchers.

B. Quantum Computers. See Quantum Computing at a Tipping Point?

This change in zeitgeist makes the thought experiment proposed below much less outlandish. What, exactly, does Gork perceive? Why couldn't you be Gork? (Note that the AGI in Gork can be an entirely classical algorithm even though he exists in a quantum simulation.)




Slide from this [Caltech IQI] talk. See also illustrations in Big Ed.
Survey questions:

1) Could you be Gork the robot? (Do you split into different branches after observing the outcome of, e.g., a Stern-Gerlach measurement?)

2) If not, why? e.g.,

I have a soul and Gork doesn't!  Copenhagen people, please use exit on your left.

Decoherence solved all that!  Sorry, try again. See previous post.

I don't believe that quantum computers will work as designed, e.g., sufficiently large algorithms or subsystems will lead to real (truly irreversible) collapse. Macroscopic superpositions that are too big (larger than whatever was done in the lab last week!) are impossible.

QM is only an algorithm for computing probabilities -- there is no reality to the quantum state or wavefunction or description of what is happening inside a quantum computer. Tell this to Gork!

Stop bothering me -- I only care about real stuff like the Higgs mass / SUSY-breaking scale / string Landscape / mechanism for high-Tc / LIBOR spread / how to generate alpha. 
[ 2018: Ha Ha -- first 3 real stuff topics turned out to be pretty boring use of the last decade... ]
Just as A. and B. above have become less outlandish assumptions, our ability to create large and complex superposition states with improved technology (largely developed for quantum computing; see Schrodinger's Virus) will make the possibility that we ourselves exist in a superposition state less shocking. Future generations of physicists will wonder why it took their predecessors so long to accept Many Worlds.

Bonus! I will be visiting Caltech next week (Tues and Weds 1/8-9). Any blog readers interested in getting a coffee or beer please feel free to contact me :-)

Sunday, December 31, 2017

Happy New Year 2018


Greetings from the Central Coast of California! I've been spending part of the holiday working with the kids on their swimming. Hope to get both of them qualified for the Michigan middle school state championship meet :-)







It's hard to beat sunshine, palm trees, and an outdoor pool in December!



On the beach, New Year's Day :-)







A brief exposition on the nature of tides:



Friday, December 29, 2017

The Bonfire of the Black Public Intellectual Vanities: Economist Glenn Loury on Ta Nehisi Coates and Cornell West

Glenn Loury is Merton P. Stoltz Professor of the Social Sciences, Department of Economics, Brown University. John McWhorter is Associate Professor of English and Comparative Literature at Columbia University, where he teaches linguistics, American studies, philosophy, and music history.



(Video will start at 20:50 but the entire conversation is worth a listen.)
[20:50] ... I'm talking about 65 or 70 percent of kids born to unmarried women. You can't tell me that that doesn't matter. It matters. There could be many explanations for it, but don't try to ignore that fact. Development, the test scores? This whole edifice that we'd built of Diversity and Inclusion, it's founded on a lie, John. Because the issue is performance and the Asians have demonstrated that. The facts are so palpable that it amazes me that people can't look at them. The Asians have demonstrated -- these are people who are second generation descendants; people were born 10,000 miles from here -- it [the USA] is an open society. African-American under-representation is a reflection of African-American under-development. Now, we can go into the historical reasons for that. If the issue is who is to blame ... plenty enough blame to go around. But the fundamental imperative is to enhance the development and that won't happen unless you acknowledge the absence of it. The test scores reflect an inadequate acquisition of functional and cognitive capacities essential to functioning in the modern world and the gaps are enormous etcetera...
Now Loury gets really worked up:
[23:50] ... the Afro Studies hustle ... the avoidance of the necessity of failure against standards in order for the standards to be meaningful and for the kind of disciplines and capacities that constitute excellence to be honed and developed. It's a shell game. It's a lie, ok. That's what I'm saying. Just say that the jails are full of black people means that the criminal justice system is racist and to leave it at that when the bodies pile up in Chicago and elsewhere. To talk about Diversity / Inclusion is the way of legitimating and institutionalizing a deferential and racist withholding of judgment from African-American people to perform at the level of excellence at a place like MIT or Caltech or Brown or Columbia or Yale requires. I mean, I'm really really angry about this because people are being dishonest about this in the interest of a Coon Show, John, a Coon Show -- that's what we're talking about ...
More at [25:17] The Bonfire of the Black Public Intellectual Vanities. See earlier post Talking Ta-Nehisi Coates, Seriously?

See also Loury's Kenneth Arrow Lecture, Department of Economics, Columbia University: Persistent Racial Inequality in the US: An Economic Theorist’s Account (PDF).

Monday, December 25, 2017

Peace on Earth, Good Will to Men 2017



For years, when asked what I wanted for Christmas, I've been replying: Peace On Earth, Good Will To All Men :-)

No one ever seems to recognize that this comes from the bible, Luke 2.14 to be precise!

Linus said it best in A Charlie Brown Christmas:
And there were in the same country shepherds abiding in the field, keeping watch over their flock by night.

And, lo, the angel of the Lord came upon them, and the glory of the Lord shone round about them: and they were sore afraid.

And the angel said unto them, Fear not: for, behold, I bring you good tidings of great joy, which shall be to all people.

For unto you is born this day in the city of David a Saviour, which is Christ the Lord.

And this shall be a sign unto you; Ye shall find the babe wrapped in swaddling clothes, lying in a manger.

And suddenly there was with the angel a multitude of the heavenly host praising God, and saying,

Glory to God in the highest, and on earth peace, good will toward men.

Merry Christmas!

Two years ago today I shared the following story on this blog: Nativity 2050

For an update, see The Future is Here: Genomic Prediction in MIT Technology Review


And the angel said unto them, Fear not: for, behold, I bring you good tidings of great joy, which shall be to all people.
Mary was born in the twenties, when the tests were new and still primitive. Her mother had frozen a dozen eggs, from which came Mary and her sister Elizabeth. Mary had her father's long frame, brown eyes, and friendly demeanor. She was clever, but Elizabeth was the really brainy one. Both were healthy and strong and free from inherited disease. All this her parents knew from the tests -- performed on DNA taken from a few cells of each embryo. The reports came via email, from GP Inc., by way of the fertility doctor. Dad used to joke that Mary and Elizabeth were the pick of the litter, but never mentioned what happened to the other fertilized eggs.

Now Mary and Joe were ready for their first child. The choices were dizzying. Fortunately, Elizabeth had been through the same process just the year before, and referred them to her genetic engineer, a friend from Harvard. Joe was a bit reluctant about bleeding edge edits, but Mary had a feeling the GP engineer was right -- their son had the potential to be truly special, with just the right tweaks ...
See also [1], [2], and [3].

Tuesday, December 19, 2017

Low SES does not decrease heritability of cognitive ability (N=300k)


These researchers, from Stanford, Northwestern, and the University of Florida, analyze a large population of twins and siblings (~24k twins and ~300k children in total, born 1994-2002 in Florida). They find no evidence of SES (Socio-Economic Status) moderation of genetic influence on test scores (i.e., cognitive ability). The figure above shows the usual pattern of lower pairwise correlations in test performance between non-identical twins and ordinary sibs, consistent with strong heritability. (In figure, ICC = Intraclass Correlation = ratio of between-pair variance to total variance; SS/OS = Same/Opposite Sex.) The researchers find, via further analysis (see below), that lower SES does not decrease heritability. No large GxE effect at low SES.

Earlier work by Turkheimer and collaborators (with much smaller sample size) suggested that low SES can drastically reduce the genetic heritability of intelligence. Their result has been widely publicized, but over time evidence is accumulating against it.

Note that Economics Nobelist James J. Heckman is the editor at PNAS who handled this paper. Heckman is an expert statistician and one of the most highly cited researchers in the area of childhood education and human capital. He was also a vocal critic of The Bell Curve, but seems (now) to accept the validity of general intelligence as a construct, its heritability, and the difficulty of increasing intelligence through environmental intervention. He tends to focus on other, more trainable, factors that influence life success, such as (my interpretation) Conscientiousness, Rule Following, Pro-Sociality, etc. ("non-cognitive skills").
Socioeconomic status and genetic influences on cognitive development
PNAS doi: 10.1073/pnas.1708491114

Significance
A prominent hypothesis in the study of intelligence is that genetic influences on cognitive abilities are larger for children raised in more advantaged environments. Evidence to date has been mixed, with some indication that the hypothesized pattern may hold in the United States but not elsewhere. We conducted the largest study to date using matched birth and school administrative records from the socioeconomically diverse state of Florida, and we did not find evidence for the hypothesis.

Abstract
Accurate understanding of environmental moderation of genetic influences is vital to advancing the science of cognitive development as well as for designing interventions. One widely reported idea is increasing genetic influence on cognition for children raised in higher socioeconomic status (SES) families, including recent proposals that the pattern is a particularly US phenomenon. We used matched birth and school records from Florida siblings and twins born in 1994–2002 to provide the largest, most population-diverse consideration of this hypothesis to date. We found no evidence of SES moderation of genetic influence on test scores, suggesting that articulating gene-environment interactions for cognition is more complex and elusive than previously supposed.
From the paper. Note SS/OS = Same/Opposite Sex, SES = Socio-Economic Status.
First, Turkheimer and Horn indicate that “the between-pair variance of MZ pairs decreases in poor environments” (ref. 21, p. 63). Contrary to this relationship, we found that the between-pair variance of SS twins is actually lowest in the highest SES families. Given that SS twins are a relatively equal combination of MZ and DZ twins, one possibility is that a pattern supporting the hypothesis among MZ SS twins is masked by an even stronger pattern in the opposite direction among DZ SS twins. However, Fig. 3 shows that corresponding results for OS twins (all of whom are DZ) give no indication of such a pattern. Between-pair variances in achievement test scores for high-school educated parents of OS twins are higher in all cases than it is for parents without a high school diploma.

Second, Turkheimer and Horn report that “the within-pair variance of MZ twin pairs increases at lower levels of SES: poverty appears to have the effect of making MZ twins more different from each other” (ref. 21, p. 61). We would therefore expect in our data that the within-pair variance for SS twins whose mother did not graduate from high school would be higher than the variance for SS twins whose mother has a high school diploma. However, this is not the case in any of the SS twin comparisons shown in Fig. 3.
Via SSC -- thanks, Scott!

Added remarks about context and broader implications: This paper does not exclude SES effects on intelligence. Rather, it excludes a hypothesis (big nonlinear effect at low SES; GxE!) that has been widely discussed: In good environments individuals can achieve their full genetic potential, and consequently measured heritability is high. However, in bad environments individuals don't achieve their full genetic potential, and (perhaps) do not even realize the full effect of beneficial genetic variants, so heritability is much reduced. This reasonable sounding hypothesis is not supported by the Florida data, suggesting that genetic influence is similarly strong in both high and low SES families.

Now, just how strong is this genetic influence? Many large studies have been conducted on populations of twins (raised together and apart), adoptees (who end up resembling their biological parents much more than the adoptive parents who raised them), and ordinary siblings. The results suggest very high heritability of adult intelligence -- broad sense heritability may be as high as ~0.8!
Wikipedia: Recent twin and adoption studies suggest that while the effect of the shared family environment is substantial in early childhood, it becomes quite small by late adolescence. These findings suggest that differences in the life styles of families whatever their importance may be for many aspects of children's lives make little long-term difference for the skills measured by intelligence tests.

Monday, December 18, 2017

Quantum Computing near a Tipping Point?

I received an email from a physicist colleague suggesting that we might be near a "tipping point" in quantum computation. I've sort of followed quantum computation and quantum information as an outsider for about 20 years now, but haven't been paying close attention recently because it seems that practical general purpose quantum computers are still quite distant. Furthermore, I am turned off by the constant hype in the technology press...

But perhaps my opinion is due for an update? I know some real quantum computing people read this blog, so I welcome comments.

Here's part of what I wrote back:
I'm not sure what is meant by "tipping point" -- I don't think we know yet what qubit technology can be scaled to the point of making Shor's Algorithm feasible. The threat to classical cryptography is still very far off -- you need millions* of qubits and the adversary can always just increase the key length; the tradeoffs are likely to be in favor of the classical method for a long time.

Noisy quantum simulators of the type Preskill talks about might be almost possible (first envisioned by Feynman in the Caltech class he gave in the 1980s: Limits to Computation). These are scientifically very interesting but I am not sure that there will be practical applications for some time.

* This is from distant memory so might not be quite right. The number of ideal qubits needed would be a lot less, but with imperfect qubits/gates and quantum error-correction, etc., I seem to remember a result like this. Perhaps millions is the number of gates, not qubits? (See here.)
These are the Preskill slides I mentioned -- highly recommended. John Preskill is the Feynman Professor of Theoretical Physics at Caltech :-)



Here's a summary of current and near-term hardware capability:

Thursday, December 14, 2017

100 Billionaires In Beijing Alone



Real talk from former Australian Prime Minister Paul Keating on the strategic outlook for Australia in Asia, the rise of China, and the likely future military balance of power in the Pacific region.

More from the Australian strategic viewpoint. Balance of power in the Western Pacific.

From the YouTube transcript:
29:18 [Eventually... Total] Chinese GDP is twice as large as America's so the idea that this great massive economy is going to be a strategic client of the United States that they are kept in line by the US 7th fleet that the US 7th fleet controls its coasts six miles off the ... territorial sea is of course nonsense but this is what the Pivot was all about. This is what Hillary Clinton and Barrack Obama's Pivot was all about was about the reestablishment of US power...

... you know it's simply unreal and if we try and become remain party to that piece of nonsense you know... that's not to say we don't need the US strategically in Asia as a balancing and conciliating power we do, but if we are party to the nonsense that we will line up for the United States to maintain its strategic hegemony in Asia over China we must have troubles...

Wednesday, December 13, 2017

Nature, Nurture, and Invention: analysis of Finnish data



What is the dominant causal mechanism for the results shown above? Is it that better family environments experienced by affluent children make them more likely to invent later in life? Is it that higher income fathers tend to pass on better genes (e.g., for cognitive ability) to their children? Obviously the explanation has important implications for social policy and for models of how the world works.

The authors of the paper below have access to patent, income, education, and military IQ records in Finland. (All males are subject to conscription.) By looking at brothers who are close in age but differ in IQ score, they can estimate the relative importance of common family environment (such as family income level or parental education level, which affect both brothers) versus the IQ difference itself. Their results suggest that cognitive ability has a stronger effect than shared family environment. Again, if one just looks at probability of invention versus family income or SES (see graph), one might mistakenly conclude that family environment is the main cause of increased likelihood of earning a patent later in life. In fact, higher family SES is also correlated to superior genetic endowments which can be passed on to the children.
The Social Origins of Inventors
Philippe Aghion, Ufuk Akcigit, Ari Hyytinen, Otto Toivanen
NBER Working Paper No. 24110
December 2017

In this paper we merge three datasets - individual income data, patenting data, and IQ data - to analyze the determinants of an individual's probability of inventing. We find that: (i) parental income matters even after controlling for other background variables and for IQ, yet the estimated impact of parental income is greatly diminished once parental education and the individual's IQ are controlled for; (ii) IQ has both a direct effect on the probability of inventing an indirect impact through education. The effect of IQ is larger for inventors than for medical doctors or lawyers. The impact of IQ is robust to controlling for unobserved family characteristics by focusing on potential inventors with brothers close in age. We also provide evidence on the importance of social family interactions, by looking at biological versus non-biological parents. Finally, we find a positive and significant interaction effect between IQ and father income, which suggests a misallocation of talents to innovation.
From the paper:
... IQ has both a direct effect on the probability of inventing which is almost five times as large as that of having a high-income father, and an indirect effect through education ...

... an R-squared decomposition shows that IQ matters more than all family background variables combined; moreover, IQ has both a direct and an indirect impact through education on the probability of inventing, and finally the impact of IQ is larger and more convex for inventors than for medical doctors or lawyers. Third, to address the potential endogeneity of IQ, we focused on potential inventors with brothers close in age. This allowed us to control for family-specific time-invariant unobservables. We showed that the effect of visuospatial IQ on the probability of inventing is maintained when adding these controls.

More on the close brothers analysis (p.24).
We look at the effect of an IQ differential between the individual and close brother(s) born at most three years apart.16 This allows us to include family fixed effects and thereby control for family-level time-invariant unobservables, such as genes shared by siblings, parenting style, and fixed family resources. Table 4 shows the results from the regression with family-fixed effects. The first column shows the baseline OLS results using the sample on brothers born at most three years apart. Notice that we include a dummy for the individual being the first born son in the family to account for birth-order effects. The second column shows the results from a regression where we introduce family fixed effects. We lose other parental characteristics than income due to their time-invariant nature.17 The main finding in Table 4 is that the coefficients on "IQ 91-95" and "IQ 96-100" [ these are percentiles, not IQ scores ] in Column 2 (i.e. when we perform the regression with family fixed effects) are the same as in the OLS Column 1. This suggests that these coefficients capture an effect of IQ on the probability of inventing which is largely independent of unobserved family background characteristics, as otherwise the OLS coefficients would be biased and different from the fixed effects estimates.

Note Added: Finland is generally more egalitarian than the US, both in terms of wealth distribution and access to education. But the probability of invention vs family income graph is qualitatively similar in both countries (see Fig 1 in the paper). The figure below is from recent US data; compare to the Finland figure at top.


Thanks to some discussion (see comments) I noticed that in the Finnish data the probability of invention seems to saturate at high incomes (see top figure, red circle), whereas it continues to rise strongly at top IQ scores (middle figure above; also perhaps in the US data above?). It would be interesting to explore this in more detail...

Friday, December 08, 2017

Recursive Cortical Networks: data efficient computer vision



Will knowledge from neuroscience inform the design of better AIs (neural nets)? These results from startup Vicarious AI suggest that the answer is yes! (See also this company blog post describing the research.)

It has often been remarked that evolved biological systems (e.g., a baby) can learn much faster and using much less data than existing artificial neural nets. Significant improvements in AI are almost certainly within reach...

Thanks to reader and former UO Physics colleague Raghuveer Parthasarathy for a pointer to this paper!
A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs

Science 08 Dec 2017: Vol. 358, Issue 6368, eaag2612
DOI: 10.1126/science.aag2612

INTRODUCTION
Compositionality, generalization, and learning from a few examples are among the hallmarks of human intelligence. CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart), images used by websites to block automated interactions, are examples of problems that are easy for people but difficult for computers. CAPTCHAs add clutter and crowd letters together to create a chicken-and-egg problem for algorithmic classifiers—the classifiers work well for characters that have been segmented out, but segmenting requires an understanding of the characters, which may be rendered in a combinatorial number of ways. CAPTCHAs also demonstrate human data efficiency: A recent deep-learning approach for parsing one specific CAPTCHA style required millions of labeled examples, whereas humans solve new styles without explicit training.

By drawing inspiration from systems neuroscience, we introduce recursive cortical network (RCN), a probabilistic generative model for vision in which message-passing–based inference handles recognition, segmentation, and reasoning in a unified manner. RCN learns with very little training data and fundamentally breaks the defense of modern text-based CAPTCHAs by generatively segmenting characters. In addition, RCN outperforms deep neural networks on a variety of benchmarks while being orders of magnitude more data-efficient.

RATIONALE
Modern deep neural networks resemble the feed-forward hierarchy of simple and complex cells in the neocortex. Neuroscience has postulated computational roles for lateral and feedback connections, segregated contour and surface representations, and border-ownership coding observed in the visual cortex, yet these features are not commonly used by deep neural nets. We hypothesized that systematically incorporating these findings into a new model could lead to higher data efficiency and generalization. Structured probabilistic models provide a natural framework for incorporating prior knowledge, and belief propagation (BP) is an inference algorithm that can match the cortical computational speed. The representational choices in RCN were determined by investigating the computational underpinnings of neuroscience data under the constraint that accurate inference should be possible using BP.

RESULTS
RCN was effective in breaking a wide variety of CAPTCHAs with very little training data and without using CAPTCHA-specific heuristics. By comparison, a convolutional neural network required a 50,000-fold larger training set and was less robust to perturbations to the input. Similar results are shown on one- and few-shot MNIST (modified National Institute of Standards and Technology handwritten digit data set) classification, where RCN was significantly more robust to clutter introduced during testing. As a generative model, RCN outperformed neural network models when tested on noisy and cluttered examples and generated realistic samples from one-shot training of handwritten characters. RCN also proved to be effective at an occlusion reasoning task that required identifying the precise relationships between characters at multiple points of overlap. On a standard benchmark for parsing text in natural scenes, RCN outperformed state-of-the-art deep-learning methods while requiring 300-fold less training data.

CONCLUSION
Our work demonstrates that structured probabilistic models that incorporate inductive biases from neuroscience can lead to robust, generalizable machine learning models that learn with high data efficiency. In addition, our model’s effectiveness in breaking text-based CAPTCHAs with very little training data suggests that websites should seek more robust mechanisms for detecting automated interactions.

Wednesday, December 06, 2017

AlphaZero: learns via self-play, surpasses best humans and machines at chess


AlphaZero taught itself chess through 4 hours of self-play, surpassing the best humans and the best (old-style) chess programs in the world.
Chess24: 20 years after DeepBlue defeated Garry Kasparov in a match, chess players have awoken to a new revolution. The AlphaZero algorithm developed by Google and DeepMind took just four hours of playing against itself to synthesise the chess knowledge of one and a half millennium and reach a level where it not only surpassed humans but crushed the reigning World Computer Champion Stockfish 28 wins to 0 in a 100-game match. All the brilliant stratagems and refinements that human programmers used to build chess engines have been outdone, and like Go players we can only marvel at a wholly new approach to the game. ...
ArXiv preprint:
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.
Excerpt:
Finally, we analysed the chess knowledge discovered by AlphaZero. Table 2 analyses the most common human openings (those played more than 100,000 times in an online database of human chess games (1)). Each of these openings is independently discovered and played frequently by AlphaZero during self-play training. When starting from each human opening, AlphaZero convincingly defeated Stockfish, suggesting that it has indeed mastered a wide spectrum of chess play.

Blog Archive

Labels