Enhancement
Make store content search engine accessible
Issue description
The content of the store is currently not accessible for hypertext search engines, because the links between pages are not sent in the response, but are only rendered via JavaScript. While the theoretical impact of this has (REST, follow your nose) is a serious issue, the currently urgent implication is that a search engine cannot index the content of a store. It should be possible to find the store content on google.
Developer comments
This seems to be more complicated than expected, since search engine vendors, consider the varying of the representation for their spiders as "cloaking" and might blacklist sites that expose such behavior. Additionally the great number of links between pages might be considered an attempt to improve the ranking by illegal means. These harsh consequences are understandable because dark forces used such methods to trick search engines. Yet, it is very unfortunate because creating a HTML only presentation of every page for spider would be an optimal solution.
A sitemap (a list of URLs and some additional information submitted separatly to the search engine vendor) will not help, since this only incereases the entry points for the spider, but does not avoid the visiting of the page.
Maybe a sitemap for humans might help to expose at least the most important topics and their names and where to find more information about that topic.
One possibility is to drop the necessity of the client to have a topic map engine and just server out HTML content (using the "rel" attribute of a link). The downside of this would be that the client does not have the power to merge two topic maps rendering the include/transclude feature dysfunctional.
I think the best solution is to create a minimal html representation, which is augmented or replaced by the JTM. This has an impact on performance though.
I implemented the solution with a html representation and linking from the start page in order to make the content crawlable.
|
|
We are sorry
This page cannot be displayed in your browser. Use Firefox, Opera, Safari, or Chrome instead.