No, take it to a vet. If you're lucky they can just be lanced but if the
infection is deep seated already which it sounds like it is then your kit
will need anaethetic while the wounds are flushed.

Trying to do this yourself will probably result in un-necesary suffering for
your poor kitten - why don't you want to take it to a vet?

The wound probably needs to be debrided. If it gets bad enough, the
kitty will end up having to have a drain put in it. I've seen cats who
healed on their own from abscesses just fine, once they drained. But
I've seen them turn very, very nasty, too.  It would be better to pay
the vet now to clean the wound properly than pay later to have a wick
or a drain put in it and have to make repeat visits.
Vets generally won't sell antibiotics OTC without seeing the animal.

Sherry