OATH-Frames: Characterizing Online Attitudes towards Homelessness via LLM Assistants

Homelessness in the U.S. is widespread, eliciting complex attitudes among individuals (e.g. critical as well as sympathetic), often expressed on social media. These attitudes are challenging to summarize at scale, further obfuscating the broader public opinion which advocacy organizations use to guide public policy and reform efforts. Our work proposes an approach to enable a large-scale study on homelessness via two major contributions. First, with the help of domain experts in social work, we characterize Online Attitudes towards Homelessness in nine hierarchical frames: OATH-Frames on a collection of 4K social media posts. Further, in an effort to ease the annotation, we employ GPT-4 as an LLM assistant to the experts; GPT-4 annotation presents an attractive trade off owing to a 6.5x speedup in annotation time despite only incurring a 2 point F1 difference in annotation performance. Our effort results in a collection of 8K social media posts labeled by domain and trained experts (with and without GPT-4 assistance). Second, using predicted OATH-Frames on a Flan-T5-Large model trained on our data, we perform a large-scale analysis on 3.1M posts on homelessness. We find marked differences in perceptions towards homelessness between the east and west coast of the U.S. We also find that posts often pit people experiencing homelessness, specifically veterans, against immigrants and asylum seekers as being either more- or less-deserving of resources and aid.