In this paper I reject the claim – made both by Tamar Szabo Gendler in On the Epistemic Costs of Implicit Bias and Jennifer Saul in Scepticism and Implicit Bias – that in order to be epistemically and morally responsible, social categories should not influence our evaluations of individuals or subsequent actions. I will provide evidence against the claim by denying its empirical plausibility, emphasizing the epistemic and moral benefits that may come from social categories, and reconceptualizing the inclusion of base-rate information. Throughout the paper I will emphasize the unique interlocking of epistemic and moral considerations that are relevant to implicit bias, bias mitigation, and responsibility. It is my hope that this analysis lays the groundwork for an account of the right ways social categories can affect our judgments, i.e. the ways in which such influence may improve our epistemic and moral situations rather than degrade them.