It’s a trivial difference, but I can hazard guesses on both the fandom and personal levels.
On a personal level, as a kid who identified with Hermione and also loved the color blue, it was another “Hermione is like me!” moment and bb!me was excited to see the dress in the movies in the same way that bb!me wanted to see Hermione arguing with Umbridge over finishing the DADA book or the way that bb!me wanted to see the Ravenclaw Common Room on-screen (…hell, adult me wanted to see the ‘Claw common room on-screen.) So as a personal reaction, it bugs me on that level - a minor/unimportant change, yes, but something that bugs me on a silly gut-level/emotional/out of proportion to logic level.
(On a personal level, the Ravenclaw color and mascot change bothers me much more - it fucked up all the official merchandise forever and that really makes me sad.) But, yeah, that’s definitely a “just me” reason and not “fandom” reason - and of course I can only speculate on greater fandom, because obviously I am not the the spokesperson for Greater Fandom.
I can hazard a hypothesis on why this bothers Greater Fandom, though. On a larger level, the dress change is emblematic of the movies’ larger issue of changing things for no good reason - of change for change’s sake, not because it furthered the story, streamlined the plot, or helped the adaptation. I think it’s become a larger symbol of ways that the movies bothered people, going beyond making the excisions and changes that are needed for a successful adaptation. On a micro level, it’s representative of the way that filmmakers tended to add, alter, and otherwise change things for no particular reason. Two other examples that come to mind easily are shrunken heads and the failure to talk about the origins of the Marauder’s Map - they’re both tiny alterations in and of themselves, utterly trivial issues that movie-only people wouldn’t care about or miss, but that when taken with a dozen other tiny things begin to add up and end up becoming emblematic of the larger problems with the films.
I know, it seems a stretch to go from “the dress should be BLUE!” to thigns like confetti Voldemort, turning Ron into comic relief, making Hermione more flawless/less human, larger disappointments with adaptation choices (particularly in the endings of later movies - the GoF/maze sequence, OotP/DoM sequence, HBP/no First Battle of Hogwarts, DH/Confetti Voldemort), choosing to excise things like Snape’s Worst Memory in OotP or the Pensieve flashbacks of HBP, and even the way that the end of DH part 2 left some movie-only people thinking that Snape was actually Harry’s father - but with larger changes, you can argue that they have some reason or advantage. In most cases, you can make a case for the change, maybe even understand that a HBP battle sequence doesn’t matter when DH2 is a huge battle sequence or that the GoF maze sequence as written might stretch on for too long and end up breaking the budget. You might even be able to argue something like “movie audiences will find the written ending of DH2 anticlimatic!” and understand the origins of Confetti Voldemort, or argue the cuts to Pensieve flashbacks as things that would throw the pace of the movie and introduce new storylines too late in the game. But there’s really no possible rhyme or reason to change something as tiny and trivial as Hermione’s dress color, and so it’s easier, in some ways, to rally behind something as trivial as a pink dress. You can argue the merits of many of the larger changes, the change in dress color is something that really has zero merit. It’s not necessarily negative, but you can’t argue a positive from it, either - and again becomes a symbol of movie-makers changing things simply because they could, not necessarily because it would improve the storyline, which is something that book fans in general are going to resent.
Basically, I think it seems to bother people out of all proportion to the size of the change because it is a pointless change - and it’s a pointless change that is unarguably pointless, and in being thus becomes representative of the larger changes that bother people so much.
Just my thoughts, of course - other people might be able to articulate other reasons why they hate that particular change.