For our group project, we undertook a detailed evaluation of GPT-4’s performance on American Mathematics Competition 12 (AMC 12) problems, utilizing two distinct prompting techniques: persona-based prompting and chain-of-thought prompting. For the persona-based approach, we initiated the conversation with the following system prompt: “You are a PhD graduate in Applied Mathematics from the University of […]

Project Report: Multimodal Evaluation of LLM’s Standardized Test Performance For the final project for our Perspectives on LLMs class, Fady Adal, Lihao Sun, Yushu Qiu and I (Jiaming Feng) worked together to design and evaluate GPT-4’s capability to answer standardized test questions that involve multimodal capabilities. Fady did background research on the topic; Lihao built […]

Title: Can large language models understand and evaluate hostile attribution biases? Introduction In history, machines have been built to be cold, unbiased, and objective. Cognitive biases, a systematic way of distorting one’s perceptions and judgments that give rise to individual’s subjective experiences, was thought to be an attribute unique to human beings. Among all the […]

Pre-preprint is available at: https://drive.google.com/file/d/1UYbU_txAfmuvWOyHUT2rWG9fNyzdOZtd/view?usp=sharing 1 Introduction The increasing prevalence of Large Language Models (LLMs) and Artificial Intelligence (AI) in contemporary society has led to the emergence of a wide array of evaluation benchmarks. Despite numerous efforts, we are faced with ever-expanding datasets encompassing a diverse range of tasks. The prevailing assumption that an increase […]

In the past couple of years, AI has been thrust into the spotlight especially with the release of ChatGPT in 2022. Some educators, professionals, and corporations are increasingly interested in using the technology to their advantage, while others are raising concerns over the technology’s limitations and dangers. Related to this, one question that our group […]

MultiModal-Eval: Project Report For a rendered pdf with the figures check here: https://people.cs.uchicago.edu/~fady/llmreport.pdf MultiModal-Eval: Project Report Fady Adal December 8, 2023 1 Introduction Since the release of ChatGPT by OpenAI at the end of November 2022, there has been a lot of debate and myths surrounding what it is capable of. One persistent claim, and […]

My final project for our class, Perspectives on Large Language Models, was focused on developing a curriculum for students to learn how to use large language models (LLMs) to learn about language variation and engage with individuals who speak different variations of the same language. We were motivated to make a project on language variations […]

Click here to see full document with table and appendix. Consider this scenario: you are a wildlife photographer who left your camera set up in a nature reserve unattended. While you were gone, a rare monkey came up and snapped a cheeky photo of himself. Is this photo yours or the monkey’s? This is the […]