Technology

How to use htmlq to extract content from HTML files on Linux

There is no greater hope in human life than survival. At the same time, it is important to live well. With that in mind, every content on the website is written so that a person can get all the information from here to start his life to make beautifully.

According to that, Technology is one of the topics. It is also a part of life. Read carefully Details of Technology related article

How to use htmlq to extract content from HTML files on Linux

Most of us use love and use the jq command. It works on Linux or Unix-like systems to extract data from JSON documents. Recently I found htmlq, which is like jq and written in Rust lang. Imagine being able to sed or grep for HTML data. We can search, slice, and filter HTML data with htmlq. Let us see how to install and use this handy tool on Linux or Unix and play with HTML data.

What is htmlq tool?

It is like jq, but for HTML. Uses CSS selectors to extract bits of content from HTML files. In CSS, selectors are used to target the HTML elements on our web pages that we want to style. For example, we can extract the images or other URLs using this tool easily.

Installing htmlq on Linux or Unix

Here is how to install cargo and rustc on Ubuntu or Debian Linux using the apt command/apt-get command:
sudo apt install cargo
Then you would run:
cargo install htmlq

macOS installing cargo

Open the Terminal app and then run the port command as follows:
sudo port install cargo
Or you can install Homebrew on macOS to use the brew package manager as follows:

brew install rustup # installs both cargo and rustc
rustup-init
rustc --version

FreeBSD intall cargo

I am going to use the pkg command as follows to install rustc:
sudo pkg install rust
See how to install Rust for other operating systems. Now that I have both rustc and cargo tools, then I type the following simple command to get htmlq on my development system:
cargo install htmlq

Have you installed Rust lang? Now install htmlq for fun and profit using the cargo command.

Setting up your PATH

Make sure you add $HOME/.cargo/bin to your PATH variable to be able to run the installed binaries using the export command

Patreon supporters only guides 🤓

    • No ads and tracking

 

 

    • Join my Patreon to support independent content creators and start reading latest guides:

 

Join Patreon
# sh/bash/ksh etc
export PATH="$PATH:$HOME/.cargo/bin" 
 
# tcsh/csh etc
setenv PATH $PATH:$HOME/.cargo/bin

How to use htmlq to extract content from HTML files on Linux or Unix

Let us use the curl command to find part of a page by ID:
curl -s url | htmlq '#css-selector'
curl -s url2 | htmlq '#css-selector'
curl -s https://www.cyberciti.biz/faq/ | htmlq --pretty '#content' | more

Click to enlarge

Let us find all the links in a page. For example:
curl -s https://www.nixcraft.com | htmlq --attribute href a

Getting help

Simply run:
htmlq --help

htmlq 0.0.1
Michael Maclean <michael@mgdm.net>
Runs CSS selectors on HTML
 
USAGE:
    htmlq [FLAGS] [OPTIONS] <selector>...
 
FLAGS:
    -h, --help                 Prints help information
    -w, --ignore-whitespace    When printing text nodes, ignore those that consist entirely of whitespace
    -p, --pretty               Pretty-print the serialised output
    -t, --text                 Output only the contents of text nodes inside selected elements
    -V, --version              Prints version information
 
OPTIONS:
    -a, --attribute <attribute>    Only return this attribute (if present) from selected elements
    -f, --filename <FILE>          The input file. Defaults to stdin
    -o, --output <FILE>            The output file. Defaults to stdout
 
ARGS:
    <selector>...    The CSS expression to select

Summing up

The htmlq is a lovely tool indeed, and I liked it very much. Do check the Github source code. Try it out and let me know what you like about it in the comment section below.

 


ADVERTISEMENT
 

 

Did you like this article?
Share it on any of the following social media channels below to give us your vote. Your feedback helps us improve.

Other related Technologies ideas you might enjoy

Related Articles

Leave a Reply

Your email address will not be published.