It's hard to say. With or without hostages, October 7 was extremely traumatic; and came in the context of a population already primed to be suspicious of Palestinians. In particular, the West Bank ethnic cleansing was already well underway with the tacit support of the general population; as although for most people that support was more about apathy than proactive support. Looking at how the US lost its shit for decades after 9/11, it is clear that hostages are not necessary for that to happen. Israel has also to deal with follow up attacks, which has a way of keeping trauma fresh.
Regarding the role of the hostages in this case, the first thing to acknowledge is that the actual response by Israel has not prioritized the hostages. Critical members of Israel's current governing coalition have threatened to leave over prior attempts at a hostage deal. This has lead a serious rift developing between the current government and many of the hostage families.
However, from a propaganda side, the hostages have been a major assesset to the current government (both internationally and domestically). Most people are simply not that engaged in politics. We have heard repeatedly from Israeli military leadership that there are no achievat military goals left in Gaza. However, it is hard for that message to break through when the other side can point to the hostages and say "freeing those people is our goal". Nevermind the fact that everyone paying attention knows that military action is not an effective tool of hostage release [0] and almost all of the freed hostages have been freed as a result of diplomacy.
[0] It can be useful for leverage in negotiations; but Israel is well past the point needed for that.
Most of October 7 was a crime, even without the hostages. Taking the hostages was itself a crime, and continuing to hold them continues to be a crime.
The question of what Hamas "should" do is more complicated. Clearly following international law is not a priority for them, so that justification goes out the window.
In terms of actually advancing their interests, I don't see much benefit to them. Their biggest asset in Israeli domestic politics are the hostages. The political pressure in Israel to free them is real, and the decision makers all know that a deal is the only way to meet that. Further, a not insignificant portion of the population oppose the war in it's current form specifically because of the hostages. The only wins Hamas has gotten has been through hostage negotiations.
In exchange for giving all of that up, Hamas gets a slight benefit in the PR war. It is a very hard sell to say that is a good trade.
If you want Hamas to free the hostages, you need to get to a point where "Hamas should free the hostages" is true from the perspective of Hamas. Then, you can work on convincing them it is true. The good news is that Hamas is very amenable to the idea that releasing hostages is in their interest. That is the entire reason you take hostages: to get some benefit by releasing them.