Deeper posts from inside receivers can be problematic, especially if the offense has the match up advantage. It's going to typically be their slot vs. one of your safeties. If they get time to cross the width of the field a window will almost always open unless you commit your user to under cutting it. You almost certainly had other responsibilities within the play call, so now someone else is open underneath.
If they have a slot apprentice they can also set up double deep crosses. This by itself can be guarded (you need to pick up the crosser going to the strong side, your weak side safety can get the other) but if they start using motion or are flipping their formation after the huddle breaks we'll probably need to play something else.
The worst is probably corner routes. The weird thing is it's not all corner routes. If I play Palms against a stock Bench Play from a balanced offensive set I'm probably ok. If I play Palms (or Quarters) against Double China or any Smash that uses a shallow in for the pull down route and I'm in big, big trouble.
Shallow-and-in can also be a problem even with the needed adjustments. For example, if they are killing me with a back side slant but I don't want to get out of this defense I'm going to bring the safety way down into the box and I'm going to press on the outside. Even if it all goes right you can still cough up the completion if your press gets beaten, the WR has great catching, etc.
Without the press you can easily lose quickly on any shallow in-cut. Try a play with two verts up the same seam and a shallow route from the outside receiver underneath them.
Even simple drags can start to test your patience and discipline because it's very, very hard to play them aggressively without opening up space behind you.