Automating alignment research is one approach to alignment that has gained much more visibility with the Open AI’s superalignment plan announcement. Some of automating alignment’s selling points (if successful) include potentially resulting in an enormous amount of alignment research even in short calendar time and the apparent relative ease of (even if non-robustly) aligning systems similar to the current state-of-the-art (e.g. large language models, foundation models; see this presentation of mine for many more details), which could be used as automated alignment researchers/research assistants. Notably, if successful, automating alignment research could plausibly be the most scalable alignment research agenda (and probably by a wide margin). At the same time, strategies to automate alignment research like the superalignment plan have received a lot of criticism within the alignment community (see e.g. this post).

This project aims to get more grounding into how promising automating alignment research is as a strategy, with respect to both advantages and potential pitfalls, with the superalignment plan as a potential blueprint/example (though ideally the findings would apply more broadly). This will be achieved by reviewing, distilling and integrating relevant research from multiple areas/domains, with a particular focus on the science of deep learning and on empirical findings in deep learning and language modelling (see my presentation for examples of what this might look like/for a potential starting point). Depending on team members’ profiles, this could expand much more broadly, covering e.g. reviewing and distilling relevant literature from AI governance, multidisciplinary intersections (e.g. neuroscience and alignment), relevant predictions on prediction markets, and the promise of automating larger parts of AI risk mitigation research (e.g. including AI governance research). 

This could also inform e.g. how promising it might be to start more automated alignment/AI risk mitigation projects or to dedicate more resources to existing ones. 

See this document with more details.