The default default rule is to deny all incoming. So if you haven’t changed the default rule, then simply deleting the rule in question should close the port again.
Now all you have to do is the following:
ufw status numbered | grep "] 22"
Find the number within the [brackets] that is assigned to the 22 ALLOW IN and don’t forget it. Then:
ufw delete 123
replacing 123 with the number you remember from the previous command.
Repeat the process for the 22 (v6) ALLOW IN rule.