The petitioners in the case could file either a motion or a petition for a an extraordinary writ of mandamus, for example, both of which are allowed by the court rules, on seeing four dissents from the denial of certiorari. The rule of four is custom, but it is such longstanding custom that the Supreme Court might well hear your motion or your case--although then Supreme Court procedure would be what they care about, which is a really hard position from which to argue that the case below should be reversed.
Note, btw, that the rule of four isn't as effective as it used to be because of the cert pool.