Thoughts and Questions

Ideas I am thinking about.

Please reach out with critiques, comments, or pointers to related work (here, via email, or twitter).

Other longer essays: Choosing a Scientific Field

Machine Learning Research

How can we engineer memory and logic in language models?

How can we take a first principles approach to interpretability?

Is the field of ML focused on the right metrics for success?

What can be done on smaller ML models as proof of principle that still applies for larger models? 

Alternatives to the attention mechanism and transformers?

What are the minimal building blocks for general intelligence?

Effects of Intelligent Machines

On science and the scientific process

On education

On culture

On energy

On value

On what it means to be human

Science and Progress

What is ultimately valuable in society, and what systems generate this value?

Learning from history: why does science work? 

Tools for science: how can society encourage independent thought, and doing important work? 


Gathering better information

Ideas on how to fix problems