These days everyone seems to be speaking about Big Data - but what does it really mean? The term is used quite ambiguously in a variety of situations. For the purposes of this article, and the series, we will refer to big data whenever we mean ‘a large amount of textual data, in any format (for example plain ASCII text, XML, HTML, or any other human-readable or semi-human-readable format). Some techniques shown may work well for binary data also, when used with care and knowledge.
This is a companion discussion topic for the original entry at https://linuxconfig.org/big-data-manipulation-for-fun-and-profit-part-1