Coffee grounds that have not been brewed are acidic. When brewed, the acid goes with the coffee in the pot, leaving the grinds very close to neutral. Use only brewed coffee grinds in the garden bed unless you want to lower the acidity around a plant. Also, never mix into the soil, just sprinkle on top under the base of the plant. Doing it this way will not tie up nitrogen in the soil. the worms will come to the surface and pull it under for you. I have been using coffee grinds in the garden bed for many many years. I also bury fish remains from cleaning in between the rows and always get a bumper crop of healthy tomato and pepper plants. So far this year I have buried close to 200 pounds of fish heads. guts and carcasses in the garden bed.Coffee grounds are acidic, and peppers tend to like mildly acidic to neutral soil, so that may account for the effect you see.
Remember, plants can't uptake nutrients very efficiently if the soil pH is wrong, so if your soil is highly alkaline, adding the coffee grounds may not help the nutrient value much, but may greatly help the pH.