Flight following along complex route

[N601RX]

I've had a controller refuse to cancel flight following before.  This has happened twice and was a military controller both times.  I was in an area that I was not required to talk to anyone, but has heavy military flight training.  I assumed he knew about traffic ahead that I didn't and I did not argue about it.