As prime minister, Boris Johnson did not do enough to tackle racial inequalities