There is already a tool in the Elastic Stack to index network data into Elasticsearch: Packetbeat. Packetbeat can be configured to seize network packets live as well as read packets from a seize file with the I option. It can appreciate and parse a number of program level protocols akin to HTTP, MySQL and DNS, as well as regular flow advice. However, it’s not built for full packet catch and parsing of the myriad different protocols out in the area and is best used for monitoring actual purposes.
Especially its capacity to match responses with their usual requests and indexing the merged event is terribly useful if you’re shopping at genuine protocols. Raw packet data comprises an extraordinarily great amount of fields. As discussed above Wireshark knows about 200,000 individual fields. Most likely, the overwhelming majority of those fields is not really searched or aggregated on. Consequently, developing an index on all these fields is typically not the proper thing to do. In fact, since a big number of fields can slow down both indexing and query speed Elasticsearch 5.
5 limits the number of fields in an index to 1000 by default. Also, the output of tshark T ek contains all field values as strings, despite regardless of whether the data is actually text or numbers adding timestamps and IP addresses, for example. Without the correct data types, you are not able to carry out type real operations on these fields e. g. deciding the common packet length.