Disclaimer: I am not yet a professional researcher. But you can guess if I will be by participating in [this](TODO: link) manifold market!

Written to clarify my thoughts, and to get meta level feedback on the direction I’m sailing towards.

Luckily, most of the thoughts are not my own

Good artists copy, great artists steal

and I definitely think that (foundational) research is art.

The goal: Describe a toolkit, advice, other parts, that let one do excellent research.

Let’s look at some sources, and then drill down to condensing.

Something I said recently

i have currently made “develop research taste” mid-term goal. i think the important work in AI safety that isn’t being done is going to be hard to aim for, so one needs to develop a good aim.

I think in some way it is same question as how to aim for ‘useful’ basic research in other fields of science (because basic research’s outputs are also illegible). Many people believe the process of what basic research is valuable is random / not predictable ex ante, but I think this is not true and that good material can help you develop this sense.

Some material I have read and found helpful and had recommended by people doing (IMO) good research on AI safety:

Richard Hamming’s “You and your research” https://www.cs.virginia.edu/~robins/YouAndYourResearch.html Holden Karnofsky’s texts on researching wicked problems (He’s the guy who founded givewell): https://www.cold-takes.com/minimal-trust-investigations/, https://www.cold-takes.com/learning-by-writing/, https://www.cold-takes.com/useful-vices-for-wicked-problems/ A TLDR (Claude helped me compile this): Work on important problems, not just tractable ones. They diverge, and most people (and funding!) drifts toward tractable. (Hamming) Build your own models from primitives instead of deferring to field consensus. Taste emerges from doing this enough times that you start noticing where consensus is load-bearing vs. lazy. (minimal trust investigations) Commit to a position before researching, then update against what you find. The act of committing forces you to notice actual confusion vs. performed uncertainty. (learning by writing) Some traits that look like flaws: stubbornness, overconfidence, dismissing objections, are productively load-bearing on problems where consensus is wrong or absent. (useful vices) Taste is trainable. The “is good research predictable ex ante” question dissolves once you treat aim as a skill rather than a property of the work.

Oh, and one more modern tip: If you have questions like “I wonder if someone has researched ‘X’ yet, or researched ‘X’ using this specific perspective / tool / looking at a specific implication”, I think nowadays it’s way faster to put this to an LLM to research than to search google scholar / arxiv / whatever manually.

The components of (good) research

impact
- action-guidingness
  - for you
  - for the field of science
  - for practical use