Thoughts and Questions

How can we engineer memory and logic in language models?

• After seeing a fact once, how can we save it forever to be recalled months later? Gradient descent doesn’t seem up to the task. Is an external memory module and targeted recall sufficient?

• Can an external memory be used to recall logical principles, and encourage their use? 

How can we take a first principles approach to interpretability?

• Can we identify and force the flow of information in neural networks?

• Can we use this control to make interchangeable components to quickly test and build larger architectures? 

• Are there equivalence classes of neural networks that can be determined for each problem and each environment (ie. a representative network structure to solve a certain classification task on a certain dataset)?

Is the field of ML focused on the right metrics for success?

• Can we use past breakthroughs in ML to redesign metrics for success, to go beyond pursuing incremental advancement on quantitative benchmark tasks?

• Are there targets that are hard to achieve and important, but easy to demonstrate once complete?

What can be done on smaller ML models as proof of principle that still applies for larger models? 

• Can we take small language models, and redesign their architecture to achieve significantly improved performance?

• To study emergence of complex behavior, can we break apart increasingly difficult tasks into intermediate learning goals, and see how these emerge over time?

Alternatives to the attention mechanism and transformers?

• What is the role of hierarchy of concepts in language models?

• Is there a more efficient alternative to byte pair encoding?

• Can chain-of-thought prompting be algorithmically encoded into a language model?

• Is there a superclass of learning targets beyond next token prediction that can be used to encourage more faithful reasoning?

What are the minimal building blocks for general intelligence?

• My nascent intuition is that the building blocks are few, and might be rather abstract: the ability to reason and analogize (1), including some external memory (2), and a reinforcement learning-based decomposition of more complex ideas and problems (3).

• It may be possible to bootstrap the behavior of smaller language models to solving more complex tasks through such an external memory architecture, similar to chain of thought reasoning, or generalized policy iteration in the AlphaGo zero algorithm.

On science and the scientific process

• What will being a scientific researcher look like in the next 5-10 years with machines manipulating knowledge in more complex and accessible ways? 

  • Where can scientists create value?

  • Can we use modern machine learning tools to measure and reward different individual’s contributions to an end product, like the sequence of events that forms the seed for a company?

On education

• What disruptions to the education system are coming with machine learning, large language models, and knowledge organization software?

  • The university system grants degrees at a single moment in time to certify knowledge. Will this be replaced with continuous validation and learning against a standard expert knowledge graph?

On culture

• Can we break down barriers between cultures and foster a sense of shared responsibility and identity using emerging technology?

  • Such as through virtual reality meetings between individuals in different cultures, with real-time language translation?

On energy

• Are there latent energy sources in the environment that are accessible through measurement and feedback?

  • Such as tracking chaotic fluctuations in water waves, air currents, or electromagnetic waves, and adjusting how the energy extraction mechanism interacts with those systems?

On value

• Can we build a tool to measure and compensate various negative externalities that are not currently addressed? Is this even valuable?

On what it means to be human

• Can deep learning revolutionize the way we interact with animals?

  • First by decoding the communication intent of animals, and second by feeding back on this communication to enable more complex reasoning and behavior.

  • Are there latent gains to be made in unlocking the capabilities of animals, similar to the increase in capabilities that human children undergo through education?

• How will our views of what it means to be human shift as machines become increasingly capable in domains we considered unique to humans?

What is ultimately valuable in society, and what systems generate this value?

• My notion of value: individuals having more time, resources, and knowledge to decide how to live their lives.

• Hypothesized sources of value: 

  • Level 1 (direct sources): raw resources and human labor. These sources can be directly used. 

  • Level 2 (multipliers): knowledge/technology and accumulation of capital/machinery. These multipliers indirectly affect value, by increasing the efficiency of direct resources.

  • Level 3 (accelerators): the scientific process of discovering knowledge, and efficient networks to distribute investment capital. These systems  act through level 2 on level 1.

Learning from history: why does science work? 

• What social mechanisms have encouraged the beneficial development of society over the last 10 millennia?

• At what level of technical detail are most transformative ideas generated? Are they generated by cross-domain semi-experts, or deep experts on a topic?

• Is scientific progress stagnated or inefficient?

• To what degree is doing science more complicated today than it was in the past? Are our teaching and learning systems keeping up with the explosion of information?

Tools for science: how can society encourage independent thought, and doing important work? 

• Can we build a more efficient organizational structure to incentivize idea creation?

• Can we build tools to better help people decide what problems to work on and fields to work in? 

  • Example: measure how scientists move between research fields based on history of authorship in various domains on ArXiv. Sustained change from field 1 to field 2 counts as a preference vote. This measures individuals who have actually experienced both fields.

• Are there alternatives to the patent system to encourage innovation?

Gathering better information

• How can randomized trials inform social and public policy, or research on scientific methods?

  • Could approval of a randomized trial on a public policy become a third default option on ballots, that people can select if they are ambivalent about a policy?

Ideas on how to fix problems

• How can society avoid the problem of ideologically-triggered reasoning biases in politics?

  • Even individuals who score highly on numerical reasoning, still fail to spot factual errors when presented with ideologically-triggering tasks. See "Motivated Numeracy and Enlightened Self-Government" by Kahan et al (2017).

• How to simplify the healthcare system, for example by disentangling the different players (individuals, hospitals/doctors, and insurance companies)?

• How can we design a reformed tax and welfare system that is acceptable to all parties?