As for duplicate content, Google gets confused when you create and publish articles with similar content, and this eventually leads to indexation issues. Keyword cannibalization happens when the owner focuses his effort on ranking for a particular keyword from several different pages. When this happens, Google won’t acknowledge multiple pages; they’ll only focus on the best one thus making the other ones useless and inaccessible to search engines.
3. Finally, there's just good old research through trends and news. Google Trends, keeping up on industry news of the business, and even newsjacking (if there are relevant topics). These all require different resources depending on the business, but once you find the leaders in their news you can not only leverage them for keyword research but also glean insights into how you can become an industry leader yourself (and dominate SEO).
Yes solid keyword data dramatically changes the game as far as rankings go. I have seen many tools that provide Cost per click stats and meaningless charts that confuse most users. the data Jaaxy provides helps drive “organic” traffic to your site on a consistent basis which will out perform PPC sites every time. I like how you say thinking like the end user, as I feel that way when I search. I dive into what people are looking for and it really helps drive content idea’s. I have seen great results from that.