"There are worms all through my garden" usually turns out to mean one of two completely different things, and mixing them up leads a lot of home gardeners to either worry about something harmless or ignore something that's actually eating their crop.
A true earthworm has a soft, segmented body with visible rings running its length, no legs at all, and is usually a pinkish-brown colour. It lives in the soil itself rather than on top of leaves, and it moves by contracting and stretching its body rather than crawling with legs. Finding plenty of earthworms while digging a bed is a genuinely good sign. They tunnel through soil, improving drainage and aeration as they go, and they break down organic matter into castings that feed the soil. More earthworms generally means healthier, more workable soil, not a problem to fix.
What people usually actually mean when they say "worms are eating my plants" is caterpillars, not earthworms at all. Caterpillars have a segmented body too, but unlike an earthworm they have true legs near the head end and soft prolegs further back, and they live on foliage and fruit rather than in the soil. That's the quickest way to tell the two apart: check whether it has legs and whether you found it in the soil or on the plant itself.
That's the quickest way to tell the two apart: check whether it has legs and whether you found it in the soil or on the plant itself.
Leafroller caterpillars are one of the more common culprits in home gardens, rolling or webbing leaves together and feeding inside the shelter they've made. Cabbage white butterfly caterpillars are the classic brassica pest, stripping soft leaves on broccoli, cabbage, cauliflower and kale, which is why netting young transplants against the adult butterfly is standard practice for that crop group. In northern regions, guava moth larvae are a growing problem, tunnelling into developing fruit on trees like plum and feijoa rather than into vegetable foliage.
Once you know which one you're looking at, the response is different for each. Earthworms don't need managing at all; if anything, adding compost and organic matter to the soil encourages more of them over time. Caterpillars are a genuine pest to act on: netting susceptible crops before damage starts, checking leaf undersides regularly, and hand-picking caterpillars off small plantings are all standard, low-effort responses once you've confirmed that's what you're dealing with.
So the short version: worms in the soil, doing nothing visible to your plants, are worth celebrating. "Worms" chewing visible holes in leaves or tunnelling into fruit are caterpillars, and worth a closer look and a plan.