Predictive modeling has the potential to enhance human decision-making. However, many predictive models fail in practice due to problematic problem formulation in cases where the prediction target is an abstract concept or construct, and practitioners need to define an appropriate target variable as a proxy to operationalize the construct of interest. Selecting an appropriate proxy target variable is a challenging process in practice, requiring both domain knowledge and iterative data modeling. While emerging prototyping tools promise to accelerate this process, it remains unclear how rapid iterations influence human judgment in problem formulation. In this work, we conducted a controlled user study (đ= 48) to investigate the impact of human-machine teaming on proxy target selection. We instantiate a system offering three recommendation strategies: Relevance-First (prioritizing conceptual alignment), Performance-First (prioritizing model performance), and Pareto-Front (considering both). We find that while rapid iterations can significantly improve exploration efficiency, they also tend to amplify a âperformance biasâ: the tendency to favor well-performing proxy targets even when they are not aligned with the modeling goal. However, systems that explicitly estimate and communicate the relevance of proxy targets can mitigate this bias. Our study highlights the risks and opportunities of human-machine teaming in operationalizing machine learning target variables, yielding insights for future research to explore the opportunities and mitigate the risks.