Has there ever been consideration of adding support to limit text searching to specific columns, as in a structured log file?

[carehart]

Folks, I've just discovered ugrep--and it's a marvelous tool. SO rich, with so many capabilities and options. Thanks to all involved.

But I was wondering if there's ever been consideration of adding support to limit text searching to specific columns, as in a structured log file. I have searched the docs, the site, the wiki, and the issues/discussions/pull requests here but find none (not even closed ones).

Of course, that may well point to a deficit in me for even thinking to ask the question. Still, since you offer this Q&A discussions section, I thought I'd ask.

To be clear, I do realize that one might be able to craft a regex. And perhaps the presumption is that if one wants column-based searching, they'll workout a regex and not burden the tool with the work. Fair enough...though not everyone would readily think of a regex as a solution, and still others may find them challenging.

I simply wondered if there may be provision for things like indicating that the file being searched (or one/s within an archive) might be a structured log with some fixed columns, with well-defined columns separators and string delimiters, which might be specified as a configuration for the search.

Since the tool has so (SO!) much flexibility in SO many other ways, it really did surprise me to find that this was not only not an option but not seemingly ever discussed. Again, I throw myself on the mercy of the court if the failing is mine to even ask the question, but there you have it. :-)

Thanks for any thoughts at all, even if they may trickle in only over days, weeks, or months. I hope the discussion may help future readers who may find it.

[genivia-inc]

To anyone, please feel free to chime in.

In my opinion, awk or gawk is a great tool for similar use cases, e.g. when columns are delimited with certain characters (like commas in CSV).

There is also a unix tool called cut to cut out selected portions of each line of a file. This is very useful to cut portions that can then be searched. It accepts a list of character (column) positions or byte positions, delimiters when fields are delimited, and more. The selected columns with cut can be piped to (u)grep to search. This shows matches in the selected columns only, i.e. the rest of the line is lost. Since line numbers can be output with (u)grep option -n, those numbers can be used in principle to find the line in the input that was matched.

[carehart]

Thanks for that (I started the thread). I do appreciate such other tools could be piped together. :-)

My motivation in asking was that frankly that could be said of any of the hundreds of cool features within ugrep. (For any who think it's a basic tool, see the help, the web site, and more. It's RICH with literally several dozen features.)

And since using it for log analysis seemed very much a typical use case, I simply thought I'd raise the idea just like those past ones had been considered and added.

And I do throughly appreciate that "being an open source project, anyone can propose a pr". I'm not skilled to do that development. But even someone who IS would want to know the idea is open to consideration before starting the work. :-)

Someone else might say, "don't expect free work to be done for you". Fair enough. I'm open to considering a price to fund this effort. Interested folks can reach me directly here via github. But please do not let this discussion die on the vine assuming someone will come forward to me. I'll not be surprised if there are no takers. The offer is for the maintainers, too, of course.

Again, thanks for a great tool, which I look forward to sharing news of--even if this idea is not adopted. :-)

[carehart]

Btw, I just connected the dots that you are in fact the maintainer/creator. Again, thank you for your efforts. And my offer still stands.

But now that I realize the above was your response to my query, I sense the odds of your even wanting to add this as a funded feature might be diminished. Still I had to ask about the potential enhancement: a boy can have dreams. :-)

[carehart]

And let me add one more clarification. Perhaps if I'd stated it in my original post, it may have better explained why I was asking.

To me what makes ugrep so great (beyond its speed and rich functionality) is the distinctive ability to search archives, easily and quickly. And it's THAT which turned me onto it, as I (like many) need to search structured logs that have been archived/zipped.

It's extremely valuable that ugrep lets one search such archives WITHOUT need of extracting the files. And for those who've not tried it, this aspect too is lightning fast (in my experience so far).

And so THAT is why I didn't think crafting a bunch of piping among standard linux cli tools (while powerful) was the best solution for this use case. I'd be frequently doing the searches across different archives and in different repositories of them.

One more thing: I'd be showing others how to leverage it and they may not only be Linux lightweights, but they may even be running ugrep on Windows--without any Linux emulating capabilities.

That's why if somehow ugrep might add a bit more to facilitate search structured logs "by column" could make it all the more valuable to perhaps a new audience. Again, I'm looking forward to turning many on to it just for the fast archive searching as-is! :-)