Looking up

Long time no post, very busy times. And although I would have dearly loved to talk about ecology, in the current flux of news and articles I’m reading I feel obliged to make a post about *drumrolls* AI *crash cymbal* in research, and astrophysics. Oh wait. So this IS inevitably going to be, to some extent, about ecology and research. Read on.

Avant-propos. First, most of this post is not going to be very original, I warn you. These days, what remains of the good old www if full of critical stuff about AI, and this one is not going to be any different, I am even going to link to some of this (and encourage you to read because I only link to good, or appalling stuff). So if you’ve already overdosed on AI discussions lately (and that’s perfectly understandable), please ignore. You can skip to the last paragraph “The future” and the few tl;drs in bold if you just want the essence of this personal take. Second, who am I to speak about AI and its moral use (or non-use) in research ? tl;dr: someone who is in a pretty reasonable position to look at the big picture. More specifically, I am a a) tenured (astro)physicist b) an ecologist and nature lover c) a human-being with a conscience and strong social solidarity values. While I am not a computer scientist stricto sensu, I am also one of these people who has used (I’d say excessively in retrospect, although I’m a small player by today’s standards) big computers and sophisticated codes that I write to do science (what we call High Performance Computing, HPC). I am also decently educated in the field of statistics, model fitting and have been embedded in a massive instrumentation and data-oriented astro institute for 20 years, in different research teams. So overall, let’s say I am reasonably educated about what computers do, how we use them in research, what astrophysics and STEM research today is about, how much energy and natural resources we use for research and in the broader world, how our socio-economic systems treat people and the planet, and how research institutions and research bandwagons work. Finally, if you think the entire thing is confusing and confused, you are probably right. To paraphrase Keynes poorly, I do not care much about the details as long as the foundations are right, and I think they are. In fact, this blog post being impressionistic merely reflects that AIs today (excessively) affect pretty much every aspect of my profession, and my own existential, epistemological, and social thinking. All the little things I talk about below are essentially part of the same, bigger picture.

To the meat and potatoes. First, what do I mean by AI in the context of this post ? Essentially (but not exclusively), generative AI, that is large language models-powered software like ChatGPT, Claude etc. Emily Bender and Alex Hanna have already written the definitive early critical book about it, check it out. AIgs for instance generate what looks like sensible human-intelligible outputs (text, images etc.) in response to prompts (everyday questions like what is the average size of lizard testicles, could you write this useless evaluation report for me, who are the musicians behind Angine de Poitrine, what is Moll-dur, generate a python script to visualize this data or, to take a recent example, write code that breaks one of the main utility unix programs to back-up data). But I will also more broadly comment critically on the excessive gratuitous use of common machine learning and automation techniques in data science/astro.

The technical and epistemological case against AIs in fundamental research

tl;dr: AI and ML in fundamental research fields like astrophysics are vastly overhyped. They are intrinsically limited by our own current knowledge limitations, and their true technical benefits and merits for research are much more modest than currently advertized/sold. Also, developing these techniques in data-driven science projects increasingly tends to become the (sophisticated) self-justifying end-goal of many projects instead of the science itself.

For a good part, AIs use a sophisticated statistical black-box machinery and certainly do not “reason”, either in the casual sense that you attribute to this word when you have an interaction with another human, or in the sense that neuroscientists understand the brain (or not). Other forms of machine learning used in a lot of research these days, often dubbed AI too, do not reason either. Very broadly speaking, all these methods rely on the statistical detection of patterns and statistical trends to make a guess about an output, given some task (spot a cancer cell in an image, ask Alexa to give me the average size of lizard testicles while I’m writing this post) given a training dataset (which, in some cases, now consists of pretty much the whole stolen corpus of human knowledge and production). [I’m happy to be corrected by specialists iif that fundamentally changes the essence of the argument and conclusions below].

So what are all these tools technically good at by construction, in broad terms again ? Interpolation on their training dataset. This can be very useful in some technical contexts, and explains why ML has become a tool of choice in a data-driven and data-overwhelmed world. Given a lot of data and real-world patterns, ML algorithms can be extremely good at spotting, identifying, classifying or singling them out statistically. But note that, in the case of AIgs/LLMs working with language patterns, the language plausability that the technique delivers offers no guarantee at all that the sentences produced by the software are factually correct or logically sound (which is, I would argue, a desirable property when you interact with a human being, but the wide appeal of these tools appears to challenge my worldview of what people find desirable), and even less a guarantee of originality/novelty considering that, well, garbage in, garbage out. The lack of novelty/originality is not a problem for many applications, where this kind of technology is used as an expedient and efficient interpolation tool, but it becomes more problematic when it comes to conceptual progress in research, as I will argue later. The commercial promise then, dubbed “scaling”, is that the more data these algorithms can be trained on (bigger corpus), and the larger their capacity to spot and retain more and more patterns (bigger computers and memory), the better and more accurate the model will be, and the more sophisticated the output (“PhD level intelligence”, if not more, is the absurd gimmick used to commercially engrain this idea in our minds). Let me get this straight : Scaling up to this level of performance is currently at best wishful thinking, at worse a big lie. LLMs in particular have terrible terrible scaling, which means that reaching their stated commercial objectives would (at best) require many orders of magnitude larger corpus of original material to train on than humanity has produced so far, and also orders of magnitude larger data centers and energy than the world can produce and afford environmentally. All of this, still without any firm guarantee of factual accuracy, mind you.

Réciproquement, what are typically the things that AIs are not only bad at in practice, but also not good at all for by design, both today and in a world with 10 million times more energy or knowledge available ? Extrapolation and originality. You can fit all the data you want, and even look for patterns hidden to the human eye in data, but when you start using ML tools outside of the narrow domain in which they can be succesfully fine-tuned to operate (and they can be in many practical, specific narrow technical application examples as I mentioned earlier), there is simply no reliable way to know whether their outcome can be trusted (something I weakly understand is dubbed “zero-shot capacity to generalize outside of the training distribution”). Because blind extrapolation is not synonymous with finding truth, it is synonymous with faith, something science has been trying to separate itself from for centuries. To assess truth, you would need to discover the real thing first by yourself (hard if not impossible in many cases), and then compare to the AI model output to check it (easy). Which largely defeats the purpose of using the tool (mostly used for acceleration/short-cut purposes) in the first place. In other words, there is no such thing as a free lunch, with AIs or otherwise.

I am already ready to die on this first hill, although it is not even the global maximum of the landscape as we shall see. You can’t use ML/AIs extensively to make creative, conceptual breakthrough-grade research in wildly uncharted territory (which is my own working definition of doing interesting fundamental research). And no, by “conceptual” I don’t mean making advanced python packages to visualize, reduce or fit complex data. I know this is increasingly considered the pinnacle of research sophistication and an end in itself in data-science and astro these days, but that’s not what I am talking about at all. No, I’m talking about exploring and discovering new physics, or processes in regimes that computers are currently not able to probe if you program them with the trusted good old physics equations we know, for instance. Bad luck, as this sublime distillate of unknown things, amongst all the small and large unknown things of the world, is precisely what we should care about when we are doing fundamental research.

The non-trivial-extrapolation and originality impotency of AIs brings me to a key point regarding their use in research. If you want to do something useful and which can be trusted with such tools, in physics/astro for instance, at the very least you need your approach to be first and foremost physics-driven. First, just because you have a fancy new sophisticated fitting method (a hammer), there is still no point hyper-adjusting it at 1% precision a shitty, back of the envelope model with very simplified physics (a fly), as large as your dataset is. Not all research questions flies mandate a ML supercomputer-grade hammer to be smashed, and in fact most do not, far from that. Our understanding of most systems is so crude that aiming for statistical precision-fitting when you don’t even have a clue about the rough picture to lowest order simply doesn’t make sense. As John Tukey, of FFT-invention fame, apparently once put it ““An approximate answer to the right problem is worth a good deal more than an exact answer to an approximate problem”. Besides, a ML learning approach, by itself, has by construction exactly zero chance of revealing the usually intricate unknown physical complexity that’s missing in basic phenomenological models often used in astro. At best, it’s failure will tell you you are missing something, but so do good old classic statistical methods (like simple multivariate linear fitting) usually. Alas, an increasingly large number of astronomers have jumped on this bandwagon and are following exactly this path everyday. New ML-tech available ? Let’s revisit everything with it. This, in general, provides plenty enough justification for a research grant proposal with clear deliverables, in an epoch where the time available in research for insightful, creative thinking increasingly coincides with the time it takes to reach for a ChatGPT prompt. Who cares whether this approach makes sense scientifically, as long as the research gets funded.

Second, if you are reasonable and your ML use is carefully physics/concept-driven in the first place, I am afraid that your ambitions and dreams of Stockholm are inevitably going to take a serious hit. A physics/concept-driven approach relies first and foremost on human hard work, knowledge, experience and human-informed model design, and then makes use of statistical, or numerical methods in a secondary, auxiliary way to make further, and usually limited and largely predictable-in-advance progress if you have a basic idea of the phenomenology of your system (which is what I mean by physics-driven). Typically, when it is well-used, a new numerical technique, whatever it is, helps you to push the limits just a little bit beyond what would have been possible otherwise, while keeping control and understanding of the whole scientific process and reasoning at all steps. Small gains and improvements is how most research works behind the scenes. This is useful, but also different, and much more limited in scope and ambition than expecting to get a whole new theory using a ChatGPT or Claude prompt. Alas, here too it seems that some colleagues have drunk the Kool-aid (the anecdotic optimistic theoretical physics use case reported in that article is concerned with LLMs, but there is no paucity of absurd use of standard ML techniques in my specific area of expertise, astrophysical fluid dynamics and plasma physics, although I am not inclined to show the many tickets I have in public).

The non-technical, and vastly more important moral case against AIs

Now, because I know part of the readership of this post will be scientists and colleagues who will fume at the previous paragraphs and rant at my technical and epistemologic worldview (which will undoubtedly be considered extreme, offensive, outdated, isolated, irresponsibly subversive to the young generation etc.), I want you to forget everything about the previous conversation and look at the broader picture. tl,dr: using technologies whose very existence requires, or indirectly legitimates the exploitation of a sizeable fraction of the resources of the world, destroys the environment, minds, education and humanity, is not worth it, independently of their ability to deliver research results. More bluntly: as a scientist interested in the wonders of the natural world, you shouldn’t possible drink, and defend the use of that poisoned well for the sake of merely improving knowledge to the margins. Yes, even if that implies searching for a different job, or working in a different field that is less prone to using this “tech”. Research, and advancing knowledge, is not a worthy endeavour at all costs in the world we are living in. This is the bigger hill, and absolute maximum of the landscape which I am ready to die on if I survive the bandwagon assault on the first.

A few weeks ago, my colleague David Hogg pointed out, in an essay posted on arxiv, that the reason why we are doing astrophysics (and I would argue why we are thinking more broadly) is much more about the satisfaction we find in the process of doing it than about the results themselves (you can think of the purpose of philosophy and dialectics too if you’ve had enough of astro at this stage). From there he concludes that using AIs to do most of our jobs (as people in his own research institute are attempting to do with Denario, see this piece in Science for more details) is somewhat problematic and undesirable (there’s more to his piece than that and I am not doing justice to his more nuanced views and arguments). I concur with this, and think David’s essay rightly captures an important aspect of the problem, although merely one of self-interest and self-preservation of our mental health. That, whatever the quality of its outputs, and whatever the field, AI is stealing away from us many of the gratifying and essential human things about being a researcher/creative thinker in the first place. I will add to this, as a keen observer of the march of academia, that this is obviously neither a concern for the people deciding of science policy, nor for an increasing number of senior researchers despite being already in stable, tenured positions. This is already very telling of the shakiness of academic moral foundations, if you ask me.

Now, for me this is just a minor part of the problem. I couldn’t care less if many colleagues prefer to effectively nuke their own brain and deliberately trash the hard-to-earn academic reputation of rigour of their scholar field with the electronic equivalent of hard drugs (I can’t even write doping as there is not even a guarantee of genuine, significant advancement of knowledge) allowing them to get quick, seemingly technologically advanced answers to increasingly marginal problems in the grandscheme of things (astro is back in my mind). I couldn’t care less about the practice of vibe-researching, and the broader, unjustified systematization of computationally expensive ML approaches if this did not have broader, extremely negative social and environmental effects.

Even just a few years ago, I still held the view that most scholars, of all human people and despite their own shortcomings, perverse professional incentives, and dubious values in some quarters (hi, eugenist geneticists), would have taken a step back after being offered to use a technology that represents an existential threat to their own objects of study (the world), and to their own future (not in terms of being replaced by computers, but rather in terms of encouraging the development a technology that destroys societies that enable things like human curiosity to thrive and to be recognized as important in the first place). If anything, today’s piece in Science on AI in astrophysics definitely settles the matter. If the most important thing to debate in this community, whose whole reason of existence is to (checks note) *look at the world*, is whether AI is going to be a game-changer or a mere practical convenience to write reports and articles, taking the money and silicon chips happily handed out by NVIDIA, the “shovel-seller” (and only profit-maker so far) of the new gold rush in the process, with just a few minor side-effect inconveniences to egos and to how well we sleep at night (many academics are already on antidepressants anyways), then the train of moral values has already well and truly left the station. It was already bad enough that an increasingly large number of colleagues seem ok to give a blank cheque to what Bender & Hanna call “a racist pile of linear algebra” (also dubbed “automated mansplaining” I’ve heard recently) to vibe-research. That they can’t even look beyond the academic rat-race and short-term benefits and instant-computer gratification provided by this pseudo-, harmful tech is another level. I should not have been surprised though. This is the natural extension, and in fact the culmination of years of astrophysics turning into an ever more expensive and polluting data- and phd student-grinding industrial research complex for ever more marginal gains. If this feels to you like the decadence of a field that was once one of the biggest and most respectable empires in science, you are not alone.

I could speak about, and get animated for days about the various detailed moral, environmental, social and ethical problems coming with the use of AI and the mindless reflex of automation of research, in astro and elsewhere in research and of course way beyond research. I could give you detailed numbers and specific illustrations of all that is wrong with this approach, how in the end society (education, health care, social care just to name a few) as a whole, and the poor are going to pay for the grifters that we are happy to assist in their grift by succumbing in research to their hype. Things that you have already read hundreds of time like me if you have been paying even minimal attention. The point is : it looks like we (as a profession) don’t really care about any of this because, as with climate change, we are not part of the population that is going to pay the price of this folly in the shorter-term. Research has long been a rich-country endeavour for the most part, and it comes as no surprise that even as we view ourselves as “progressives”, we are happy to indulge in the thought of using AI as long as the true costs of it are delocalized (for the time being) elsewhere (and those of AI are going to be truly existential if the tech becomes operational at the scale needed for the debates about its technical merits in research to really make sense, as right now these debates are essentially a side distraction). In astro, we now claim to be post-colonialist in our new instrument-building approaches in remote parts of the world, but at the same time we are following the exact same non-reflexive path with AI as we did for decades with big telescopes building. Some argue that “another, cheaper, more ethical AI” is possible (in French). Techno-positive mathematicians too have been quick on jumping on this pseudo-restrained bandwagon with what I can only qualify as a grandiloquent “Leiden declaration on AI and mathematics“. I think we as a scientific community are failing big, by falling into old moral smoke and mirror games again and again, justifying the unjustifiable in science by all means, deluding ourselves in self-comforting declarations of intention. If you picture an atomic bomb explosion while reading this, this is entirely normal and intended, because AI is truly the atomic weapon crisis of computer science. Not becaue of the risks of generalized AI, but because it is in the process of destroy what little remained of our sanity, deference to facts, and chances of surviving the climate and ecological crises. Some are also of course actively selling their own “better, smaller” snake oil or have started to defend their niche after other critical colleagues have recently launched an “objection de conscience” manifesto against AIg (also in French).

The future

Which finally brings me to what I think is perphaps the most important issue right now in all of this, the younger generation of students and researchers. This sentence caught most of my attention in today’s Science piece : “And of course they [students], too, were using LLMs and agents. The grad student said their peers “are trying to [be] realistic: ‘This is about to be our future, this is what our careers are going to be.’” Of course this is both true and entirely representative of the situation, I have witnessed it in my own astro research department over the last 2 years. Maybe I’m wrong and blinded by my own professional transition, but my impression is that this is already very true in some fields like astro, but not quite as much, by a long shot, in others at this stage. So, perhaps (surely) quite idealistically, I say to all the bright young astro kids who are still astonished, like I still am as an astrophysicist myself by vocation, by the incredible thing that it is to live on this planet, as one among tens of millions of other living species:

if you don’t want your professional future to turn into a senseless nightmare in this academic hellscape plague by the massive, indiscriminate use and hype for AI/ML, if you are feeling burn-out by AI and the loss of your academic ideals, please avoid using AIs and mindless automation at all costs, and know that you are very much welcome to work in fields that care more about the disastrous effects that we humans are making on the planet.

Although my own new colleagues point out that environmental scientists are not exempt of criticism and are not models of introspection themselves, I think we can be a haven grounded in reality for young people like you. In ecology for instance, we can still work in different ways in the future, closer to the matrix of life inextricably tied to our own biological human existences. Sure, this may result in fewer speculative alternative theories of gravity generated by an AI doing everything from coffee to paper writing while we stare at the emptiness of our lives and souls. Instead, you’ll get a shot at changing the world, with your professional skills and at your own modest grassroot level, one little human thought, field-data collection, heuristic model or handcrafted educational action at a time. In societies and scientific communities running high on their own privileges, that will exhaust natural resources and life on the very non-astrophysics timescale of centuries, the contradiction in terms of permanent growth in physics research, and the AI frenzy that it too has succumbed to, should and will soon be over anyways. So why not spend the next decades of your scientific life working on the path to a post-growth future instead ?

Final tl;dr: this brilliant sketch by Eleanor Morton.