تقرير
What is it for a Machine Learning Model to Have a Capability?
| العنوان: | What is it for a Machine Learning Model to Have a Capability? |
|---|---|
| المؤلفون: | Harding, Jacqueline, Sharadin, Nathaniel |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning |
| الوصف: | What can contemporary machine learning (ML) models do? Given the proliferation of ML models in society, answering this question matters to a variety of stakeholders, both public and private. The evaluation of models' capabilities is rapidly emerging as a key subfield of modern ML, buoyed by regulatory attention and government grants. Despite this, the notion of an ML model possessing a capability has not been interrogated: what are we saying when we say that a model is able to do something? And what sorts of evidence bear upon this question? In this paper, we aim to answer these questions, using the capabilities of large language models (LLMs) as a running example. Drawing on the large philosophical literature on abilities, we develop an account of ML models' capabilities which can be usefully applied to the nascent science of model evaluation. Our core proposal is a conditional analysis of model abilities (CAMA): crudely, a machine learning model has a capability to X just when it would reliably succeed at doing X if it 'tried'. The main contribution of the paper is making this proposal precise in the context of ML, resulting in an operationalisation of CAMA applicable to LLMs. We then put CAMA to work, showing that it can help make sense of various features of ML model evaluation practice, as well as suggest procedures for performing fair inter-model comparisons. Comment: forthcoming in the British Journal for the Philosophy of Science (BJPS) |
| نوع الوثيقة: | Working Paper |
| الوصول الحر: | http://arxiv.org/abs/2405.08989Test |
| رقم الانضمام: | edsarx.2405.08989 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |