Why Should I Care?
Fast-glob is a high-performance file system globbing library for Node.js that offers significant speed improvements over traditional glob implementations. As reported by npm-compare.com, fast-glob is designed to be up to 60% faster than conventional globbing libraries, making it particularly effective for tasks involving large-scale file searching and manipulation in Node.js applications.
What is Glob
Glob is a pattern-matching technique used in computer programming to specify sets of filenames or pathnames using wildcard characters. Originally developed for Unix shells, glob patterns have become a standard feature in many programming languages and file system utilities. The term “glob” is derived from “global command” in early Unix systems.Key aspects of glob include:
- Wildcard characters: Glob patterns use special characters like ’*’ (matches any number of characters) and ’?’ (matches a single character) to create flexible file matching rules
Glob patterns offer versatile functionality for managing files and directories across various operating systems and programming environments. They enable path matching, allowing for file and directory name matching over multiple file system levels, ensuring efficient searching without unnecessary traversals. Originally a Unix concept, glob patterns are now supported widely across different platforms.
Commonly applied in tasks such as file searching, build processes, and configuration files, glob patterns provide a straightforward and powerful method to identify groups of files or directories. Here are some practical uses for glob patterns:
- File Organization: Quickly move or copy specific file types using patterns like
*.jpgto select all JPEG images in a directory for sorting photos. - Backup Creation: Use patterns such as
documents/**/*.{doc,docx,pdf}to back up all Word documents and PDFs within a “documents” folder and its subdirectories. - Cleaning Up Downloads: Apply
Downloads/*.{exe,zip,tar.gz}to list or delete all executable and compressed files from a Downloads folder. - Code Searches: Developers can utilize
src/**/*.jsto locate all JavaScript files in a project’s source directory, regardless of the folder depth. - System Maintenance: System administrators might use
/var/log/*.logto rotate or analyze all log files in a particular directory. - Data Analysis: Data scientists can work with
data/**/2023*.csvto handle all CSV files from 2023 across multiple subdirectories.
These examples illustrate how glob patterns can simplify file operations for users with diverse levels of technical expertise, from casual users to professional developers and system administrators.## Who Built It Dude who built the tool works at Yandex (Russias version of Google) https://www.canonium.com https://github.com/mrmlnc Has a good list of glob repositories https://github.com/stars/mrmlnc/lists/glob
How To Setup
Instructions to Use the Script 1. Install fast-glob if it’s not already installed:
npm install fast-glob
Here are the next steps after installing fast-glob:
-
Create a script to utilize fast-glob in your project:
touch glob-script.js
-
Add the import to your script:
echo “const glob = require(‘fast-glob’);” > glob-script.js
-
Run your first glob pattern:
node glob-script.js
Simple script I ran on Katana Master Repository
// Import necessary modules
const fs = require(‘fs’)
const path = require(‘path’)
const fg = require(‘fast-glob’)
// Function to search for a word in a file
async function searchWordInFiles(directory, word) {
try {
// Find all files in the specified directory
const files = await fg(${directory}/**/*.*)
// Iterate over each file
for (const file of files) {
const content = fs.readFileSync(file, ‘utf-8’)
// Check if the file content contains the word
if (content.includes(word)) {
console.log("${word}" found in: ${file})
}
}
} catch (error) {
console.error(‘Error occurred:’, error)
}
}
// Call the function to search for ‘bookmarks’ in the ‘pipelines’ folder
searchWordInFiles(‘pipelines’, ‘bookmarks’)
File called globtest.js then I ran node globtest.js in terminal and damn!!!!
The term “universe” has been detected in the following locations:
- pipelines/generate/pipeline.py
- pipelines/generate/pipeline_it.py
- pipelines/generate/tox.ini
- pipelines/generate/universe_definitions.py
- pipelines/build/gcb/pipeline-deploy.yml
- pipelines/generate/domain/idea.py
- pipelines/generate/components/generate_trade_ideas.py
- pipelines/generate/components/ingest_bonds_per_universe.py
- pipelines/generate/components/ingest_bonds_price_history.py
- pipelines/generate/components/ingest_bookmarks.py
- pipelines/generate/scripts/run_flex_generate_trade_ideas.sh
- pipelines/generate/scripts/run_flex_ingest_bonds_per_universes.sh
- pipelines/generate/scripts/run_flex_ingest_bonds_price_history.sh
- pipelines/generate/scripts/run_generate_trade_ideas.sh
- pipelines/generate/scripts/run_ingest_bonds_per_universes.sh
- pipelines/generate/scripts/run_ingest_bonds_price_history.sh
- pipelines/generate/components/combiners/bonds_to_universe.py
- pipelines/generate/components/combiners/bonds_to_universe_test.py
- pipelines/generate/components/combiners/deduplicate_bonds.py
- pipelines/generate/components/combiners/deduplicate_bonds_test.py
- pipelines/generate/components/filters/universes.py
- pipelines/generate/components/filters/universes_test.py
- pipelines/generate/components/exports/pubsub/ideas.py
- pipelines/generate/components/exports/pubsub/universes.py
- pipelines/generate/components/exports/bigquery/bonds.py
- pipelines/generate/components/exports/bigquery/bookmarks.py
- pipelines/generate/components/exports/bigquery/prices.py
- pipelines/generate/components/exports/bigquery/trade_ideas.py
- pipelines/generate/lib/bigquery/evolutions/0001_create-bonds-table.sql
- pipelines/generate/lib/bigquery/evolutions/0002_create-dropped-bonds-table.sql
- pipelines/generate/lib/bigquery/evolutions/0003_create-prices-table.sql
- pipelines/generate/lib/bigquery/evolutions/0005_create-trade_ideas-table.sql johnjohn@imac-pro katana-master % node testglob.js The term “bookmarks” has been detected in the following locations:
- pipelines/generate/pipeline.py
- pipelines/generate/pipeline_it.py
- pipelines/build/gcb/pipeline-deploy.yml
- pipelines/generate/components/generate_trade_ideas.py
- pipelines/generate/components/ingest_bookmarks.py
- pipelines/generate/scripts/run_flex_ingest_bookmarks.sh
- pipelines/generate/scripts/run_ingest_bookmarks.sh
- pipelines/generate/components/filters/bond_pairs.py
- pipelines/generate/components/filters/bond_pairs_test.py
- pipelines/generate/components/filters/bonds.py
- pipelines/generate/components/transforms/bond_pairs_to_candidates.py
- pipelines/generate/components/sources/gcs/bookmarks.py
- pipelines/generate/components/sources/gcs/generate_trade_idea_candidates.py
- pipelines/generate/lib/bigquery/evolutions/0004_create-bookmarks-table.sql