تقرير
Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implications
| العنوان: | Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implications |
|---|---|
| المؤلفون: | Agarwal, Sandhini, Krueger, Gretchen, Clark, Jack, Radford, Alec, Kim, Jong Wook, Brundage, Miles |
| سنة النشر: | 2021 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Computers and Society |
| الوصف: | Recently, there have been breakthroughs in computer vision ("CV") models that are more generalizable with the advent of models such as CLIP and ALIGN. In this paper, we analyze CLIP and highlight some of the challenges such models pose. CLIP reduces the need for task specific training data, potentially opening up many niche tasks to automation. CLIP also allows its users to flexibly specify image classification classes in natural language, which we find can shift how biases manifest. Additionally, through some preliminary probes we find that CLIP can inherit biases found in prior computer vision systems. Given the wide and unpredictable domain of uses for such models, this raises questions regarding what sufficiently safe behaviour for such systems may look like. These results add evidence to the growing body of work calling for a change in the notion of a 'better' model--to move beyond simply looking at higher accuracy at task-oriented capability evaluations, and towards a broader 'better' that takes into account deployment-critical features such as different use contexts, and people who interact with the model when thinking about model deployment. Comment: arXiv admin note: substantial text overlap with arXiv:2103.00020 |
| نوع الوثيقة: | Working Paper |
| الوصول الحر: | http://arxiv.org/abs/2108.02818Test |
| رقم الانضمام: | edsarx.2108.02818 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |