I'll share my thoughts, having worked for LN many years ago (back when it was called Mead Data Central, starting before NEXIS even existed).
MDC, based on a project started in Ohio, began building LEXIS, libraries of state and federal court decisions and related materials. Addition of materials started slowly-but-steadily and increased speed over time.
After a few years, West saw that its hard-copy position was being threatened, so it started Westlaw. At first, its search capabilities were noticeably more feeble and its collections were more limited, but Westlaw eventually became competitive.
I believe the answer to your Q (BTW, they constitute a duopoly, rather than a monopoly) lies in the following:
- It takes a huge amount of money and time to build the extensive, integrated, organized, specifically-searchable collections that these two services have. Anyone who started trying to compete now would have an insurmountable burden.
- Contrary to your assertion, not all of the materials these services offer are available to the public [for free elsewhere].
- Lawyers who need to do research appreciate having everything (including citators such as Shepard's) in one place.
- Lawyers who use these services a lot do not mind paying for them because, directly or indirectly, those lawyers' clients end up footing the bill.
I rarely need to do legal research. Sometimes I just do Google searches. Occasionally, when I need to use one of those services (e.g., to obtain a case that I cannot find anywhere else), I do so by credit card.