المؤلفون: Chen, Xuweiyi, Ma, Ziqiao, Zhang, Xuejun, Xu, Sihan, Qian, Shengyi, Yang, Jianing, Fouhey, David F., Chai, Joyce

مصطلحات موضوعية: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Computation and Language

الوصف: Large vision language models (LVLMs) often suffer from object hallucination, producing objects not present in the given images. While current benchmarks for object hallucination primarily concentrate on the presence of a single object class rather than individual entities, this work systematically investigates multi-object hallucination, examining how models misperceive (e.g., invent nonexistent objects or become distracted) when tasked with focusing on multiple objects simultaneously. We introduce Recognition-based Object Probing Evaluation (ROPE), an automated evaluation protocol that considers the distribution of object classes within a single image during testing and uses visual referring prompts to eliminate ambiguity. With comprehensive empirical studies and analysis of potential factors leading to multi-object hallucination, we found that (1) LVLMs suffer more hallucinations when focusing on multiple objects compared to a single object. (2) The tested object class distribution affects hallucination behaviors, indicating that LVLMs may follow shortcuts and spurious correlations.(3) Hallucinatory behaviors are influenced by data-specific factors, salience and frequency, and model intrinsic behaviors. We hope to enable LVLMs to recognize and reason about multiple objects that often occur in realistic visual scenes, provide insights, and quantify our progress towards mitigating the issues.
Comment: Accepted to ALVR @ ACL 2024 | Project page: https://multi-object-hallucination.github.ioTest/

الوصول الحر: http://arxiv.org/abs/2407.06192Test

المؤلفون: Pi, Shi, Sasaki, Misao, Wang, Ao, Wang, Jianing

مصطلحات موضوعية: Astrophysics - Cosmology and Nongalactic Astrophysics, General Relativity and Quantum Cosmology, High Energy Physics - Phenomenology, High Energy Physics - Theory

الوصف: High-frequency primordial gravitational waves (PGWs) with wave numbers larger than the Hubble parameter at the end of inflation are originated from the ultraviolet (UV) modes, which are never stretched out of the horizon. Such a UV tail of the PGW energy spectrum has a spurious logarithmic divergence. We study the origin of such a divergence, and find that it comes from the instantaneous inflation-to-post-inflation transition, which can be removed by considering a finite duration. For the first time, we obtain a semi-analytical expression for the PGW energy spectrum. We find that the UV tail decays exponentially, while the decay rate depends solely on the transition rate. When there is a stiff post-inflationary stage, the enhanced PGW displays a characteristic spectral shape of power-law increasing and exponential decaying. We propose a fitting formula which can be used for signal searching.
Comment: 23 pages, 13 figures

الوصول الحر: http://arxiv.org/abs/2407.06066Test

المؤلفون: Gao, Shen, Wen, Yuntao, Zhu, Minghang, Wei, Jianing, Cheng, Yuhan, Zhang, Qunzi, Shang, Shuo

مصطلحات موضوعية: Computer Science - Computation and Language

الوصف: Most economic theories typically assume that financial market participants are fully rational individuals and use mathematical models to simulate human behavior in financial markets. However, human behavior is often not entirely rational and is challenging to predict accurately with mathematical models. In this paper, we propose \textbf{A}gent-based \textbf{S}imulated \textbf{F}inancial \textbf{M}arket (ASFM), which first constructs a simulated stock market with a real order matching system. Then, we propose a large language model based agent as the stock trader, which contains the profile, observation, and tool-learning based action module. The trading agent can comprehensively understand current market dynamics and financial policy information, and make decisions that align with their trading strategy. In the experiments, we first verify that the reactions of our ASFM are consistent with the real stock market in two controllable scenarios. In addition, we also conduct experiments in two popular economics research directions, and we find that conclusions drawn in our \model align with the preliminary findings in economics research. Based on these observations, we believe our proposed ASFM provides a new paradigm for economic research.

الوصول الحر: http://arxiv.org/abs/2406.19966Test

المؤلفون: Li, Pingzhi, Kools, Thomas J., Pezeshki, Hamed, Joosten, Joao M. B. E., Li, Jianing, Igarashi, Junta, Hohlfeld, Julius, Lavrijsen, Reinoud, Mangin, Stephane, Malinowski, Gregory, Koopmans, Bert

مصطلحات موضوعية: Condensed Matter - Mesoscale and Nanoscale Physics, Physics - Optics

الوصف: Single pulse all-optical switching of magnetization (AOS) in Co/Gd based synthetic ferrimagnets carries promises for hybrid spintronic-photonic integration. A crucial next step progressing towards this vision is to gain insight into AOS and multi-domain state (MDS) behavior using longer pulses, which is compatible with state-of-the-art integrated photonics. In this work, we present our studies on the AOS and MDS of [Co/Gd]n (n = 1, 2) using ps optical pulses across a large composition range. We theoretically and experimentally show that a large Gd layer thickness can enhance the AOS energy efficiency and maximum pulse duration. We have identified two augmenting roles of Gd in extending the maximum pulse duration. On the inter-atomic level, we found that more Gd offers a prolonged angular momentum supply to Co. On the micromagnetic level, a higher Gd content brings the system to be closer to magnetic compensation in the equilibrized hot state, thereby reducing the driving force for thermally assisted nucleation of domain walls, combating the formation of a MDS. Our study presents a composition overview of AOS in [Co/Gd]n and offers useful physical insights regarding AOS fundamentals as well as the projected photonic integration.

الوصول الحر: http://arxiv.org/abs/2406.16027Test

المؤلفون: Fu, Ying, Li, Yu, You, Shaodi, Shi, Boxin, Chen, Linwei, Zou, Yunhao, Wang, Zichun, Li, Yichen, Han, Yuze, Zhang, Yingkai, Wang, Jianan, Liu, Qinglin, Yu, Wei, Lv, Xiaoqian, Li, Jianing, Zhang, Shengping, Ji, Xiangyang, Chen, Yuanpei, Zhang, Yuhan, Peng, Weihang, Zhang, Liwen, Xu, Zhe, Gou, Dingyong, Li, Cong, Xu, Senyan, Zhang, Yunkang, Jiang, Siyuan, Lu, Xiaoqiang, Jiao, Licheng, Liu, Fang, Liu, Xu, Li, Lingling, Ma, Wenping, Yang, Shuyuan, Xie, Haiyang, Zhao, Jian, Huang, Shihua, Cheng, Peng, Shen, Xi, Wang, Zheng, An, Shuai, Zhu, Caizhi, Li, Xuelong, Zhang, Tao, Li, Liang, Liu, Yu, Yan, Chenggang, Zhang, Gengchen, Jiang, Linyan, Song, Bingyi, An, Zhuoyu, Lei, Haibo, Luo, Qing, Song, Jie, Liu, Yuan, Li, Qihang, Zhang, Haoyuan, Wang, Lingfeng, Chen, Wei, Luo, Aling, Li, Cheng, Cao, Jun, Chen, Shu, Dou, Zifei, Liu, Xinyu, Zhang, Jing, Zhang, Kexin, Yang, Yuting, Gou, Xuejian, Wang, Qinliang, Liu, Yang, Zhao, Shizhan, Zhang, Yanzhao, Yan, Libo, Guo, Yuwei, Li, Guoxin, Gao, Qiong, Che, Chenyue, Sun, Long, Chen, Xiang, Li, Hao, Pan, Jinshan, Xie, Chuanlong, Chen, Hongming, Li, Mingrui, Deng, Tianchen, Huang, Jingwei, Li, Yufeng, Wan, Fei, Xu, Bingxin, Cheng, Jian, Liu, Hongzhe, Xu, Cheng, Zou, Yuxiang, Pan, Weiguo, Dai, Songyin, Jia, Sen, Zhang, Junpei, Chen, Puhua

مصطلحات موضوعية: Computer Science - Computer Vision and Pattern Recognition

الوصف: The intersection of physics-based vision and deep learning presents an exciting frontier for advancing computer vision technologies. By leveraging the principles of physics to inform and enhance deep learning models, we can develop more robust and accurate vision systems. Physics-based vision aims to invert the processes to recover scene properties such as shape, reflectance, light distribution, and medium properties from images. In recent years, deep learning has shown promising improvements for various vision tasks, and when combined with physics-based vision, these approaches can enhance the robustness and accuracy of vision systems. This technical report summarizes the outcomes of the Physics-Based Vision Meets Deep Learning (PBDL) 2024 challenge, held in CVPR 2024 workshop. The challenge consisted of eight tracks, focusing on Low-Light Enhancement and Detection as well as High Dynamic Range (HDR) Imaging. This report details the objectives, methodologies, and results of each track, highlighting the top-performing solutions and their innovative approaches.
Comment: CVPR 2024 PBDL Challenges: https://pbdl-ws.github.io/pbdl2024/challenge/index.htmlTest

الوصول الحر: http://arxiv.org/abs/2406.10744Test

المؤلفون: Yang, Jianing, Visvanathan, Harshine, Wang, Yilin, Hu, Xinyi, Gormley, Matthew

مصطلحات موضوعية: Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Machine Learning

الوصف: Many tasks within NLP can be framed as sequential decision problems, ranging from sequence tagging to text generation. However, for many tasks, the standard training methods, including maximum likelihood (teacher forcing) and scheduled sampling, suffer from exposure bias and a mismatch between metrics employed during training and inference. DAgger provides a solution to mitigate these problems, yet it requires a metric-specific dynamic oracle algorithm, which does not exist for many common metrics like span-based F1, ROUGE, and BLEU. In this paper, we develop these novel dynamic oracles and show they maintain DAgger's no-regret guarantee for decomposable metrics like span-based F1. We evaluate the algorithm's performance on named entity recognition (NER), text summarization, and machine translation (MT). While DAgger with dynamic oracle yields less favorable results in our MT experiments, it outperforms the baseline techniques in NER and text summarization.

الوصول الحر: http://arxiv.org/abs/2406.09393Test

المؤلفون: Wang, Qizhou, Han, Bo, Yang, Puning, Zhu, Jianing, Liu, Tongliang, Sugiyama, Masashi

مصطلحات موضوعية: Computer Science - Machine Learning

الوصف: The compelling goal of eradicating undesirable data behaviors, while preserving usual model functioning, underscores the significance of machine unlearning within the domain of large language models (LLMs). Recent research has begun to approach LLM unlearning via gradient ascent (GA) -- increasing the prediction risk for those training strings targeted to be unlearned, thereby erasing their parameterized responses. Despite their simplicity and efficiency, we suggest that GA-based methods face the propensity towards excessive unlearning, resulting in various undesirable model behaviors, such as catastrophic forgetting, that diminish their practical utility. In this paper, we suggest a set of metrics that can capture multiple facets of real-world utility and propose several controlling methods that can regulate the extent of excessive unlearning. Accordingly, we suggest a general framework to better reflect the practical efficacy of various unlearning methods -- we begin by controlling the unlearning procedures/unlearned models such that no excessive unlearning occurs and follow by the evaluation for unlearning efficacy. Our experimental analysis on established benchmarks revealed that GA-based methods are far from perfect in practice, as strong unlearning is at the high cost of hindering the model utility. We conclude that there is still a long way towards practical and effective LLM unlearning, and more efforts are required in this field.

الوصول الحر: http://arxiv.org/abs/2406.09179Test

المؤلفون: Song, Xiaoshuai, Diao, Muxi, Dong, Guanting, Wang, Zhengyang, Fu, Yujia, Qiao, Runqi, Wang, Zhexu, Fu, Dayuan, Wu, Huangxuan, Liang, Bin, Zeng, Weihao, Wang, Yejie, GongQue, Zhuoma, Yu, Jianing, Tan, Qiuna, Xu, Weiran

مصطلحات موضوعية: Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Machine Learning

الوصف: Computer Science (CS) stands as a testament to the intricacies of human intelligence, profoundly advancing the development of artificial intelligence and modern society. However, the current community of large language models (LLMs) overly focuses on benchmarks for analyzing specific foundational skills (e.g. mathematics and code generation), neglecting an all-round evaluation of the computer science field. To bridge this gap, we introduce CS-Bench, the first bilingual (Chinese-English) benchmark dedicated to evaluating the performance of LLMs in computer science. CS-Bench comprises approximately 5K meticulously curated test samples, covering 26 subfields across 4 key areas of computer science, encompassing various task forms and divisions of knowledge and reasoning. Utilizing CS-Bench, we conduct a comprehensive evaluation of over 30 mainstream LLMs, revealing the relationship between CS performance and model scales. We also quantitatively analyze the reasons for failures in existing LLMs and highlight directions for improvements, including knowledge supplementation and CS-specific reasoning. Further cross-capability experiments show a high correlation between LLMs' capabilities in computer science and their abilities in mathematics and coding. Moreover, expert LLMs specialized in mathematics and coding also demonstrate strong performances in several CS subfields. Looking ahead, we envision CS-Bench serving as a cornerstone for LLM applications in the CS field and paving new avenues in assessing LLMs' diverse reasoning capabilities. The CS-Bench data and evaluation code are available at https://github.com/csbench/csbenchTest.
Comment: Work in progress

الوصول الحر: http://arxiv.org/abs/2406.08587Test

المؤلفون: Zhu, Jianing, Han, Bo, Yao, Jiangchao, Xu, Jianliang, Niu, Gang, Sugiyama, Masashi

مصطلحات موضوعية: Computer Science - Machine Learning

الوصف: Machine unlearning as an emerging research topic for data regulations, aims to adjust a trained model to approximate a retrained one that excludes a portion of training data. Previous studies showed that class-wise unlearning is successful in forgetting the knowledge of a target class, through gradient ascent on the forgetting data or fine-tuning with the remaining data. However, while these methods are useful, they are insufficient as the class label and the target concept are often considered to coincide. In this work, we decouple them by considering the label domain mismatch and investigate three problems beyond the conventional all matched forgetting, e.g., target mismatch, model mismatch, and data mismatch forgetting. We systematically analyze the new challenges in restrictively forgetting the target concept and also reveal crucial forgetting dynamics in the representation level to realize these tasks. Based on that, we propose a general framework, namely, TARget-aware Forgetting (TARF). It enables the additional tasks to actively forget the target concept while maintaining the rest part, by simultaneously conducting annealed gradient ascent on the forgetting data and selected gradient descent on the hard-to-affect remaining data. Empirically, various experiments under the newly introduced settings are conducted to demonstrate the effectiveness of our TARF.

الوصول الحر: http://arxiv.org/abs/2406.08288Test

المؤلفون: Yang, Jianing, Chen, Xuweiyi, Madaan, Nikhil, Iyengar, Madhavan, Qian, Shengyi, Fouhey, David F., Chai, Joyce

مصطلحات موضوعية: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning, Computer Science - Robotics

الوصف: The integration of language and 3D perception is crucial for developing embodied agents and robots that comprehend and interact with the physical world. While large language models (LLMs) have demonstrated impressive language understanding and generation capabilities, their adaptation to 3D environments (3D-LLMs) remains in its early stages. A primary challenge is the absence of large-scale datasets that provide dense grounding between language and 3D scenes. In this paper, we introduce 3D-GRAND, a pioneering large-scale dataset comprising 40,087 household scenes paired with 6.2 million densely-grounded scene-language instructions. Our results show that instruction tuning with 3D-GRAND significantly enhances grounding capabilities and reduces hallucinations in 3D-LLMs. As part of our contributions, we propose a comprehensive benchmark 3D-POPE to systematically evaluate hallucination in 3D-LLMs, enabling fair comparisons among future models. Our experiments highlight a scaling effect between dataset size and 3D-LLM performance, emphasizing the critical role of large-scale 3D-text datasets in advancing embodied AI research. Notably, our results demonstrate early signals for effective sim-to-real transfer, indicating that models trained on large synthetic data can perform well on real-world 3D scans. Through 3D-GRAND and 3D-POPE, we aim to equip the embodied AI community with essential resources and insights, setting the stage for more reliable and better-grounded 3D-LLMs. Project website: https://3d-grand.github.ioTest
Comment: Project website: https://3d-grand.github.ioTest

الوصول الحر: http://arxiv.org/abs/2406.05132Test