mirror of
https://github.com/kellyjonbrazil/jc.git
synced 2025-12-16 11:14:05 +00:00
Page:
Tips and Tricks
No results
15
Tips and Tricks
Kelly Brazil edited this page 2022-08-21 20:22:38 -07:00
Generic Converters
jc comes with a few generic or standard-format converters, including:
- ASCII (and Unicode) Tables
- Multi-line ASCII (and Unicode) Tables
- CSV
- INI
- Key/Value
- XML
- YAML
Sometimes commands will output in one of these supported formats for easy conversion to JSON.
Example: CURL Headers
For example, you can parse the HTTP headers from curl using the Key/Value parser:
$ curl --silent --header "origin: https://origin.example.com" \
--request GET "https://cors-test.appspot.com/test" \
--head | jc --kv -p
{
"http/2 200": "",
"cache-control": "no-cache",
"access-control-allow-origin": "https://origin.example.com",
"access-control-max-age": "0",
"access-control-allow-credentials": "true",
"set-cookie": "test=test",
"expires": "Fri, 01 Jan 1990 00:00:00 GMT",
"content-type": "application/json",
"x-cloud-trace-context": "086f2570a400102ed316c10f1719c87e",
"date": "Tue, 17 May 2022 21:39:06 GMT",
"server": "Google Frontend",
"content-length": "15",
"alt-svc": "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\""
}
Example: NMAP XML Output
Another example would be converting NMAP output to JSON. NMAP does not (yet) support native JSON output, but it does support XML output. You can use the jc --xml parser to get JSON output from NMAP:
$ nmap -sV -oX output.xml avleonov.com | jc --xml -p
{
"nmaprun": {
"@scanner": "nmap",
"@args": "nmap -sV -oX output.xml avleonov.com",
"@start": "1520760090",
"@startstr": "Sun Mar 11 12:21:30 2018",
"@version": "7.60",
"@xmloutputversion": "1.04",
"scaninfo": {
"@type": "connect",
"@protocol": "tcp",
"@numservices": "1000",
"@services": "1,3-4,6-7,9,13,17,19-26,30,32-33,37,42-43,49,53,70,79-85,88-90,99-100,106,109-111,113,119,125,135,139,143-144,146,161,163,179,199,211-212,222,254-256,259,264,280,301,306,311,340,366,389,406-407,416-417,425,427,443-445,458,464-465,481,497,500,512-515,524,541,543-545,548,554-555,563,587,593,616-617,625,631,636,646,648,666-668,683,687,691,700,705,711,714,720,722,726,749,765,777,783,787,800-801,808,843,873,880,888,898,900-903,911-912,981,987,990,992-993,995,999-1002,1007,1009-1011,1021-1100,1102,1104-1108,1110-1114,1117,1119,1121-1124,1126,1130-1132,1137-1138,1141,1145,1147-1149,1151-1152,1154,1163-1166,1169,1174-1175,1183,1185-1187,1192,1198-1199,1201,1213,1216-1218,1233-1234,1236,1244,1247-1248,1259,1271-1272,1277,1287,1296,1300-1301,1309-1311,1322,1328,1334,1352,1417,1433-1434,1443,1455,1461,1494,1500-1501,1503,1521,1524,1533,1556,1580,1583,1594,1600,1641,1658,1666,1687-1688,1700,1717-1721,1723,1755,1761,1782-1783,1801,1805,1812,1839-1840,1862-1864,1875,1900,1914,1935,1947,1971-1972,1974,1984,1998-2010,2013,2020-2022,2030,2033-2035,2038,2040-2043,2045-2049,2065,2068,2099-2100,2103,2105-2107,2111,2119,2121,2126,2135,2144,2160-2161,2170,2179,2190-2191,2196,2200,2222,2251,2260,2288,2301,2323,2366,2381-2383,2393-2394,2399,2401,2492,2500,2522,2525,2557,2601-2602,2604-2605,2607-2608,2638,2701-2702,2710,2717-2718,2725,2800,2809,2811,2869,2875,2909-2910,2920,2967-2968,2998,3000-3001,3003,3005-3007,3011,3013,3017,3030-3031,3052,3071,3077,3128,3168,3211,3221,3260-3261,3268-3269,3283,3300-3301,3306,3322-3325,3333,3351,3367,3369-3372,3389-3390,3404,3476,3493,3517,3527,3546,3551,3580,3659,3689-3690,3703,3737,3766,3784,3800-3801,3809,3814,3826-3828,3851,3869,3871,3878,3880,3889,3905,3914,3918,3920,3945,3971,3986,3995,3998,4000-4006,4045,4111,4125-4126,4129,4224,4242,4279,4321,4343,4443-4446,4449,4550,4567,4662,4848,4899-4900,4998,5000-5004,5009,5030,5033,5050-5051,5054,5060-5061,5080,5087,5100-5102,5120,5190,5200,5214,5221-5222,5225-5226,5269,5280,5298,5357,5405,5414,5431-5432,5440,5500,5510,5544,5550,5555,5560,5566,5631,5633,5666,5678-5679,5718,5730,5800-5802,5810-5811,5815,5822,5825,5850,5859,5862,5877,5900-5904,5906-5907,5910-5911,5915,5922,5925,5950,5952,5959-5963,5987-5989,5998-6007,6009,6025,6059,6100-6101,6106,6112,6123,6129,6156,6346,6389,6502,6510,6543,6547,6565-6567,6580,6646,6666-6669,6689,6692,6699,6779,6788-6789,6792,6839,6881,6901,6969,7000-7002,7004,7007,7019,7025,7070,7100,7103,7106,7200-7201,7402,7435,7443,7496,7512,7625,7627,7676,7741,7777-7778,7800,7911,7920-7921,7937-7938,7999-8002,8007-8011,8021-8022,8031,8042,8045,8080-8090,8093,8099-8100,8180-8181,8192-8194,8200,8222,8254,8290-8292,8300,8333,8383,8400,8402,8443,8500,8600,8649,8651-8652,8654,8701,8800,8873,8888,8899,8994,9000-9003,9009-9011,9040,9050,9071,9080-9081,9090-9091,9099-9103,9110-9111,9200,9207,9220,9290,9415,9418,9485,9500,9502-9503,9535,9575,9593-9595,9618,9666,9876-9878,9898,9900,9917,9929,9943-9944,9968,9998-10004,10009-10010,10012,10024-10025,10082,10180,10215,10243,10566,10616-10617,10621,10626,10628-10629,10778,11110-11111,11967,12000,12174,12265,12345,13456,13722,13782-13783,14000,14238,14441-14442,15000,15002-15004,15660,15742,16000-16001,16012,16016,16018,16080,16113,16992-16993,17877,17988,18040,18101,18988,19101,19283,19315,19350,19780,19801,19842,20000,20005,20031,20221-20222,20828,21571,22939,23502,24444,24800,25734-25735,26214,27000,27352-27353,27355-27356,27715,28201,30000,30718,30951,31038,31337,32768-32785,33354,33899,34571-34573,35500,38292,40193,40911,41511,42510,44176,44442-44443,44501,45100,48080,49152-49161,49163,49165,49167,49175-49176,49400,49999-50003,50006,50300,50389,50500,50636,50800,51103,51493,52673,52822,52848,52869,54045,54328,55055-55056,55555,55600,56737-56738,57294,57797,58080,60020,60443,61532,61900,62078,63331,64623,64680,65000,65129,65389"
},
"verbose": {
"@level": "0"
},
"debugging": {
"@level": "0"
},
"host": {
"@starttime": "1520760091",
"@endtime": "1520760120",
"status": {
"@state": "up",
"@reason": "syn-ack",
"@reason_ttl": "0"
},
"address": {
"@addr": "81.177.6.213",
"@addrtype": "ipv4"
},
"hostnames": {
"hostname": [
{
"@name": "avleonov.com",
"@type": "user"
},
{
"@name": "srv36-h-st.jino.ru",
"@type": "PTR"
}
]
},
"ports": {
"extraports": [
{
"@state": "filtered",
"@count": "936",
"extrareasons": [
{
"@reason": "no-responses",
"@count": "916"
},
{
"@reason": "host-unreaches",
"@count": "20"
}
]
},
{
"@state": "closed",
"@count": "56",
"extrareasons": {
"@reason": "conn-refused",
"@count": "56"
}
}
],
"port": [
{
"@protocol": "tcp",
"@portid": "21",
"state": {
"@state": "open",
"@reason": "syn-ack",
"@reason_ttl": "0"
},
"service": {
"@name": "ftp",
"@product": "ProFTPD",
"@method": "probed",
"@conf": "10",
"cpe": "cpe:/a:proftpd:proftpd"
}
},
{
"@protocol": "tcp",
"@portid": "22",
"state": {
"@state": "open",
"@reason": "syn-ack",
"@reason_ttl": "0"
},
"service": {
"@name": "ssh",
"@product": "OpenSSH",
"@version": "5.3",
"@extrainfo": "protocol 2.0",
"@method": "probed",
"@conf": "10",
"cpe": "cpe:/a:openbsd:openssh:5.3"
}
},
{
"@protocol": "tcp",
"@portid": "80",
"state": {
"@state": "open",
"@reason": "syn-ack",
"@reason_ttl": "0"
},
"service": {
"@name": "http",
"@product": "nginx",
"@method": "probed",
"@conf": "10",
"cpe": "cpe:/a:igor_sysoev:nginx"
}
},
{
"@protocol": "tcp",
"@portid": "443",
"state": {
"@state": "open",
"@reason": "syn-ack",
"@reason_ttl": "0"
},
"service": {
"@name": "http",
"@product": "nginx",
"@tunnel": "ssl",
"@method": "probed",
"@conf": "10",
"cpe": "cpe:/a:igor_sysoev:nginx"
}
},
{
"@protocol": "tcp",
"@portid": "2222",
"state": {
"@state": "open",
"@reason": "syn-ack",
"@reason_ttl": "0"
},
"service": {
"@name": "ssh",
"@product": "ProFTPD mod_sftp",
"@version": "0.9.7",
"@extrainfo": "protocol 2.0",
"@method": "probed",
"@conf": "10",
"cpe": "cpe:/a:proftpd:proftpd:0.9.7"
}
},
{
"@protocol": "tcp",
"@portid": "3306",
"state": {
"@state": "open",
"@reason": "syn-ack",
"@reason_ttl": "0"
},
"service": {
"@name": "mysql",
"@product": "MySQL",
"@version": "5.5.59-MariaDB",
"@method": "probed",
"@conf": "10",
"cpe": "cpe:/a:mysql:mysql:5.5.59-mariadb"
}
},
{
"@protocol": "tcp",
"@portid": "8080",
"state": {
"@state": "open",
"@reason": "syn-ack",
"@reason_ttl": "0"
},
"service": {
"@name": "http",
"@product": "Apache httpd",
"@method": "probed",
"@conf": "10",
"cpe": "cpe:/a:apache:http_server"
}
},
{
"@protocol": "tcp",
"@portid": "8443",
"state": {
"@state": "open",
"@reason": "syn-ack",
"@reason_ttl": "0"
},
"service": {
"@name": "ssl",
"@product": "Apache httpd",
"@extrainfo": "SSL-only mode",
"@tunnel": "ssl",
"@method": "probed",
"@conf": "10",
"cpe": "cpe:/a:apache:http_server"
}
}
]
},
"times": {
"@srtt": "6937",
"@rttvar": "1642",
"@to": "100000"
}
},
"runstats": {
"finished": {
"@time": "1520760120",
"@timestr": "Sun Mar 11 12:22:00 2018",
"@elapsed": "29.83",
"@summary": "Nmap done at Sun Mar 11 12:22:00 2018; 1 IP address (1 host up) scanned in 29.83 seconds",
"@exit": "success"
},
"hosts": {
"@up": "1",
"@down": "0",
"@total": "1"
}
}
}
}
Common String Conversions
jc comes with converters for common strings found in program output, files, etc. including:
- URLs
- IP Addresses
- Email Addresses
- ISO 8601 Datetimes
- Unix Timestamps
- JWT token strings
- Syslog
- CEF
Example: ISO-8601 Datetime
If your program output contains a standard ISO 8601 datetime string, you can explode all of the datetime parts to a JSON object:
$ date --iso-8601=seconds | jc --iso-datetime -p
{
"year": 2022,
"month": "Aug",
"month_num": 8,
"day": 1,
"weekday": "Mon",
"weekday_num": 1,
"hour": 5,
"hour_24": 17,
"minute": 32,
"second": 6,
"microsecond": 0,
"period": "PM",
"utc_offset": "+0000",
"day_of_year": 213,
"week_of_year": 31,
"iso": "2022-08-01T17:32:06+00:00",
"timestamp": 1659375126
}
Alternative JSON Output
Most parsers have slightly different output when using the -r (Raw) cli option. The Raw output
is typically used for troubleshooting or in cases where you don't want string to integer/float/bool
conversions or even some calculated timestamp fields.
A couple parsers will output significantly different styles of JSON if the Raw option is used:
envhistory
Example: env variations
For example, the env parser turns each environment variable into its own JSON object and adds all of these objects to a list:
$ env | jc --env -p
[
{
"name": "TERM_PROGRAM",
"value": "Apple_Terminal"
},
{
"name": "SHELL",
"value": "/bin/bash"
},
{
"name": "TERM",
"value": "xterm-256color"
},
...
]
This output makes it easy to iterate over each key/value pair and match values to predictable key names. But if you don't need that format, try the -r option:
$ env | jc --env -p -r
{
"TERM_PROGRAM": "Apple_Terminal",
"SHELL": "/bin/bash",
"TERM": "xterm-256color",
...
}
With this output there is no list to iterate over - If you know the name of the key, you can quickly get the value.
Example: history variations
The history parser is similar:
$ history | jc --history -p
[
{
"line": 1009,
"command": "jc git log | jq"
},
{
"line": 1010,
"command": "jc git log --format=full | jq"
},
{
"line": 1011,
"command": "jc git log --format=fuller --stat | jq"
},
{
"line": 1012,
"command": "jc git log --format=oneline --shortstat | jq"
},
...
]
vs.
$ history | jc --history -p -r
{
"1009": "jc git log | jq",
"1010": "jc git log --format=full | jq",
"1011": "jc git log --format=fuller --stat | jq",
"1012": "jc git log --format=oneline --shortstat | jq",
...
}
© 2019-2023 Kelly Brazil