loggrep

Package: WA2L/edrc 1.5.57
Section: General Commands (1)
Updated: 24 January 2023
Index Return to Main Contents

 

NAME

loggrep - grep in remote- or local (time-stamped) log file(s)

 

SYNOPSIS

edrc/bin/loggrep [ -h | -p ]

loggrep regex file ...

loggrep regex [[user@]host:]file ...

loggrep [ options ] [ -f "from_ts" ] [ -t "to_ts" ] [ egrep_options ] regex [[user@]host:]file ...

loggrep [ options ] [ -f "from_ts" ] [ -t "to_ts" -t "to_ts" ] [ egrep_options ] regex [[user@]host:]file ...

lgrep ...

options ::= [ -l localuser ] [ -M ] [ -s num ] [ -NUM ] [ -T id ]

 

AVAILABILITY

WA2L/edrc

 

DESCRIPTION

grep selected time range (from/to) in log files.

If the log file has time stamps, the data between a from and to time stamp is selected and greped by the specified regex.

When there are no time stamps in the log file, or the time stamp format is not recognized, the whole log file data is greped

The first 97 lines of the log file are used to resolve the time stamp format used in a particular file.

Beside text log file(s), loggrep can handle file(s) which are ziped (.zip), compressed (.Z), gziped (.gz), bzip2ed (.bz, .bz2, .bzip2) or xzed (.xz) directly.

To increase performance on scanning large log files which are bigger then 100 MByte, only the last 1'000'000 lines are analysed. To force to nevertheless analyse the whole file, specify -0 in the -NUM option.

 

FROM/TO DATA SELECTION

The -f "from_ts" and -t "to_ts" time stamps have to be specified in military format ("2021-01-18 20:57:10").

The following abbreviations and defaults apply (examples when executed on Monday, 2021-01-18):

specified from/to_tsresulting from_ts resulting to_ts
2021-01-18 00:00:002021-01-18 23:59:59
13:452021-01-18 13:45:002021-01-18 13:45:59
13:45:552021-01-18 13:45:552021-01-18 13:45:55
162021-01-16 00:00:002021-01-16 23:59:59
01-162021-01-16 00:00:002021-01-16 23:59:59
05-302020-05-30 00:00:002020-05-30 23:59:59
20202020-01-01 00:00:002020-12-31 23:59:59
2020-032020-03-01 00:00:002020-03-31 23:59:59
2020-03-272020-03-27 00:00:002020-03-27 23:59:59
2020-03-27 132020-03-27 13:00:002020-03-27 13:59:59
2020-03-27 13:452020-03-27 13:45:002020-03-27 13:45:59
2020-03-27 13:45:122020-03-27 13:45:122021-03-27 13:45:12
yesterday2021-01-17 00:00:002021-01-17 23:59:59
mon2021-01-11 00:00:002021-01-11 23:59:59
tue2021-01-12 00:00:002021-01-12 23:59:59
wed2021-01-13 00:00:002021-01-13 23:59:59
thu2021-01-14 00:00:002021-01-14 23:59:59
fri2021-01-15 00:00:002021-01-15 23:59:59
sat2021-01-16 00:00:002021-01-16 23:59:59
sun2021-01-17 00:00:002021-01-17 23:59:59

 

LOG FILE TIME STAMP FORMATS

The following time stamps are recognized (per log file):

1)
2021-01-30 23:59:59

2)
2021/01/30-23:59:59

3)
Jan 30 23:59:59

4)
---- 2021-01-30 23:59:59 ----

5)
12/16-11:34:21

6)
Nov 18, 2019 9:44:43 PM
Apr 15, 2020 10:50:51 AM

7)
End-Date: 2021-01-15 10:06:29
Start-Date: 2021-01-15 10:06:35

8)
[18/Jan/2021:10:51:23 +0100]

9)
2021/01/30 23:59:59

10)
----BEGIN: Mon Nov 18 19:51:51 UTC 2019
----END: Mon Nov 18 19:51:52 UTC 2019

11)
2020-10-27T17:44:21

12)
#1612518675

13)
2021-01-30-23:59:59

14)
[Tue Aug 27 10:56:41 2019]
[Tue Aug 27 10:56:41.1234 2019]

15)
Thu Feb 4 12:27:12 2021

16)
-80238 | Fri Feb 5 12:27:11 2021
447752 | Fri Feb 5 12:27:12 2021

17)
update-alternatives 2020-10-27 22:50:52

18)
0156 01/31/21 12:23:56
0157 01/31/21 12:24:07

19)
---- Rules Applied on Monday 2021-01-11 12:25:48

20)
[UTC 01/31/21 11:22:32]

21)
Log started: 2021-01-15 10:06:29
Log ended: 2021-01-15 10:06:35

22)
Log time: 2018-08-18 21:10:09

23)
error 2021-02-13 06:12:51
info 2021-02-13 06:12:51

24)
ERROR: apport (pid 593) Fri Feb 1 23:09:19 2019

 

OPTIONS

-h
usage message.

-p
print the timestamp formats known by loggrep, some example files where that timestamp format has been discovered and the regular expression that is used to identify the timestamp.

-l localuser
use the local user localuser to initiate the connection.

-s num
use the first num lines of the logfile for timestamp format detection.

-NUM
analyse the last NUM lines, instead of the whole file.

On large log files that exceed 100 MByte, only the last 1'000'000 rows are scanned, whereas the first 4007 lines are here used to resolve the time stamp format.


To force to scan nevertheless the whole log file, specify -0 here.

-f "from_ts"
start of data selection in military (=ISO) format, an abbreviated date or time specification as listed in the table in section DESCRIPTION or a past weekday (as: yesterday, mon, tue, wed, thu, fri, sat, sun).

Default is the current date at 00:00:00 (example:"2021-01-18 00:00:00").

-t "to_ts"
end of data selection in military (=ISO) format, an abbreviated date or time specification as listed in the table in section DESCRIPTION or a past weekday (as: yesterday, mon, tue, wed, thu, fri, sat, sun).

Default is current date at 23:59:59 (example:"2021-01-18 23:59:59").

-M
print meta data of loggrep results to file descriptor 3.

Example: 

  LOGGREP_FROM="2020-03-27 13:45:00"
  LOGGREP_TO="2020-03-27 18:00:00"
  LOGGREP_COUNT="68312"
  LOGGREP_BEGIN="2020-03-27 13:50:18"
  LOGGREP_END="2020-03-27 17:58:36"
  LOGGREP_DURATION="000:04:13:24"

This output can be used to set variables in a script: 

  #!/bin/sh

  loggrep -M -f "2020-03-27 13:45:00" -t "2020-03-27 18:00:00" \
    starting ~edrc/var/log/edrc.log 3>/tmp/meta.out 

  . /tmp/meta.out
  echo "ENTRIES: $LOGGREP_COUNT between $LOGGREP_BEGIN and $LOGGREP_END"

egrep_options
options of the egrep(1) command. If options of the egrep(1) have a second argument, specify the argument without a space (eg: specify -m5 and not -m 5).

See egrep(1) for a description of the available egrep options.

regex
regular expression PATTERN as understood by egrep respectively grep -E.

[[user@]host:]file
remote- or local file.

If a - is specified as a file option, data is read from stdin.

Beside text files, the specified file can also be ziped (.zip), compressed (.Z), gziped (.gz), bzip2ed (.bz, .bz2, .bzip2) or xzed (.xz).

 

ENVIRONMENT

$LOGGREP_FROM
from_ts as specified on the command line. However, the command line option -f "from_ts" has preference.

$LOGGREP_TO
to_ts as specified on the command line. However, the command line option -t "to_ts" has preference.

 

EXIT STATUS

0
regex is found in at least one of the listed files. no error.

1
regex is not found in any of the listed files.

2
error occured.

4
usage printed.

5
command has been aborted.

11
could not claim a temporary directory in /tmp/.

 

FILES

-

 

EXAMPLES

-

 

SEE ALSO

edrcintro(1), cat(1), edrcsetup(1m), egrep(1), grep(1), logcat(1), rcat(1), regexintro(4), ssh-exec(1), ssh-exec.cfg(4)

 

NOTES

loggrep uses rcat(1) internally to get the remote- and local files.

A call to lgrep ... is identical to a call to the loggrep ... command.

 

BUGS

-

 

AUTHOR

loggrep was developed by Christian Walther. Send suggestions and bug reports to wa2l@users.sourceforge.net .

 

COPYRIGHT

Copyright © 2023 by Christian Walther

This is free software; see edrc/doc/COPYING for copying conditions. There is ABSOLUTELY NO WARRANTY; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

 

Index

NAME
SYNOPSIS
AVAILABILITY
DESCRIPTION
FROM/TO DATA SELECTION
LOG FILE TIME STAMP FORMATS
OPTIONS
ENVIRONMENT
EXIT STATUS
FILES
EXAMPLES
SEE ALSO
NOTES
BUGS
AUTHOR
COPYRIGHT
This document was created by man2html using the manual pages.
Time: 16:53:33 GMT, August 28, 2024