Data visualizations are commonly employed to convey relationships between variables from complex datasets in exploratory data analysis. Recent advancements in Large Language Models (LLMs) have shown surprising performance in assisting data analysis and visualization. In this poster, we investigate the capabilities of LLMs for reasoning about causality between concept pairs in visualized data using line charts, bar charts, and scatterplots. By using LLMs to replicate two human-subject empirical studies about causality judgments, we how their inferences about causality between concept pairs compare to those of humans, both with and without accompanying visualizations showing varying association levels. Our findings indicate that LLMs’ causality inferences are more likely to align with human results without visualizations at very high or very low causal ratings, but LLMs are more influenced by low visualized associations and relatively unaffected by high visualized associations.