Jeffrey Shi

Intelligence / SIGINT Analyst · ETL & Data Engineering Background

Years Experience

IC Programs

B.A. Degrees

Download PDF

Support a U.S. Government / Intelligence Community cybersecurity group providing a platform for cyber assessments of systems.
Develop and maintain modularized Python ETL pipelines that parse cybersecurity scans and reports; built the Titania Nipper report-parsing module for ingest into the cyber assessment platform.
Develop scripts to clean up inefficiencies in the Postgres database and support customer-facing UI components that depend on it.
Research and develop custom Velociraptor artifacts to extract system information for determining cybersecurity compliance of assets and devices.

Python PostgreSQL Velociraptor Cyber Assessment

Supported an Intelligence Community mission group building and maintaining ETL pipelines for a live data feed into Oracle and Postgres databases for downstream customer use.
Built and maintained Python ETL pipelines on an asyncio framework, ingesting data from Kafka topics and transforming it for output into Oracle and Postgres.
Drove the migration from a Pentaho-based ETL framework to a Python- and Kubernetes-centered architecture.
Refactored and improved a custom FastAPI application feeding Neo4j, giving the customer graph visualization of all ETL processes and data flow.

Python Kafka Oracle Kubernetes Neo4j

Maintained the technical and cloud infrastructure for a U.S. Government business and data analytics team: AWS RDS running Oracle, applications on EC2, and S3 secondary storage.
Automated recurring tasks with Python web scraping — Selenium WebDriver collection formatted into pandas dataframes and written into database tables for downstream data scientists.
Ran monthly data-integrity validation across database tables via Python and SQL (cx_Oracle), correcting values flagged by the checks.

AWS Python Selenium Oracle pandas

Frontend Product Delivery Agile

Software Development