Priti Patel hit back at a letter from opposition politicians which claimed that she used her "heritage and experiences of racism to gaslight the very real racism faced by black people and communities across the UK".
Shadow communities secretary Naz Shah organised the letter in response to a statement by Ms Patel on demonstrations against racial inequality in the UK in the wake of George Floyd's killing.
She took issue with the home secretary saying at the time that "when it comes to racism, sexism, tolerance for social justice I will not take lectures from the other side of the house".
The 32 MPs who signed it told Ms Patel that "being a person of colour does not automatically make you an authority on all form of racism".
They urged her to "reflect on your words and to consider the impact it had towards black communities in the UK trying to highlight their voices against racism".
Speaking to Sophy Ridge on Sunday, Ms Patel said she was disappointed at the claims, which she rejected.
"They clearly take the stance and the position that I just don't conform to their preconceived idea or stereotypical view of what an ethnic minority woman should stand for and represent," she told Sophy Ridge on Sunday.
"That in itself is racist.
"It's very disappointing. I'm not going to dignify that letter any further."
The government has pledged to set up a Commission on Race and Ethnic Disparities to look at tackling inequalities further after the Black Lives Matter protests.
But Foreign Secretary Dominic Raab antagonised campaigners after he said he thought the "taking a knee" phenomenon started by a American footballer to protest racial injustice was actually from Game of Thrones.
He added it was a "symbol of subjugation and subordination".