Why is it scary to think about that? What feels scary about it? If he tries to force himself to think about it despite it being scary, the way his brain keeps sliding off it is "well of course we shouldn't do anything that Haru doesn't want to" and "of course the things Haru likes out of dates aren't going to be just the standard romantic tropes" and so on, which... hmm. Points at something. Especially given that, you know, it's not that picnics aren't a standard romantic trope? So that last sentence in particular is hiding something. What's it hiding?
It's that... Haru is special, and romancing Haru is not like romancing other boys, and part of that is that Haru is the kind of person who knows what he wants and goes to get it even if it's strange or unusual or nonstandard. But why did that feel relevant? What if it weren't the case that Haru was special, what if Haru did want the standard romantic tropes? Well, then he'd have accepted Jaeha's ideas, which means that if he did like them but also didn't accept them that would've meant he didn't want to do those things with Jaeha specifically. So using that particular excuse is a way for Jaeha to protect himself, his own pride. If the reason why Haru doesn't want to do these things is because he's special and needs to be romanced in a customised way, then it doesn't say anything negative about Jaeha and about their relationship. It is true that Haru needs to be romanced in a special way, of course, but the fact that it's true is what was giving that particular thought the necessary oomph to support itself as a face-saving measure.
So, Jaeha's scared that if Haru doesn't want to do those date things with him, this means that Haru doesn't actually like him. That's patently stupid.