No idea why the SQL does not work for you. It works under HeidiSQL and phpmyadmin (both with mariadb) without problems.
To your question, in this case you have to look in the spawngroup table. The Spawns are organized in groups. E.g. Spider1 33%, Spider2 33%, Bat1 33%. If that would be the case, you would have to kill the Bat1 as well as the spiders to get a 33% higher spawn chance for Spider1/2.
Edit: Found this on google about your SQL Error:
https://stackoverflow.com/questions/...ns-mysql-error