Two posts on a Friday? Crazy
If you are running the FAST 5.3 search solution in your organization for a while and things are going well, you may come to a point where someone asks you to make sure the content in the your corporate dB is exactly the same content that is available in your searchable index. This most likely will arise in a discussion when you host two or more FAST search environments or you have a Disaster Recovery site. So this brief “How-To” will help your organization keep its search data store (content) in line with the FAST searchable index. It assumes you have some knowledge about how the FAST system works and you also have some hands on experience with the product.
It is possible to dump the contents of the indexer with the stock \\esp\bin\indexerinfo tool (for help indexerinfo -h). However, it will only give you a list of internalids only and NOT
content ids.
What would need to happen is the following:
1) On your content store:
– Dump the contentid’s column from the content store table with your normal dB tools
– Then you will need to do some scripting by converting your contentid to an internalid with an available programming tool you are comfortable with.
Eg. Python loop that outputs to a file: [ PYTHON? — This is only an example, feel free to use another tool to accomplish this. ]
>>> import md5
>>> md5sum = md5.new(‘contentid’).hexdigest()
>>> print md5sum 274c14f8c673c2c1130054b957be7c3b
>>>
– Output should have contentid and internalid
2) On your FAST admin node :
– Run :: indexerinfo reportcontents collectionname > output.txt (for a particular collection that you can also parse in your store)
– or more likey – indexerinfo reportcontents > output.txt for everything…
– Run a diff (with your favorite tool) of the two output files on internalids, refeed the delta by your contentid match on the store.
There are a couple other things to consider here as well before you do this. If you are constantly feeding you may
want to pick a spot where you can suspend content flowing into the store and the index… so you can have two static
lists to compare. Otherwise you could have the store contents off from the index contents by default
~Big thanks to Nick Meader for sharing the Python piece which makes this possible
