Some problems can be solved with workflows
When we face with engineering problems, it’s too easy to fall into the trap thinking it should be solved with a technical solution. Seasoned engineers think differently, because they realize that most of the time, it’s “people” or “workflow” problems. Let me provide a few examples. Management wants analysts to use Jupyter notebook to reduce time required to create a routine report Background: Most analysts are comfortable using Microsoft Excel to work with data, some can also use SQL, but it’s rare for analysts to be familiar with Python. Jupyter notebook is an interactive development interface for data works, since users can execute a chunk of code at a time, and render data without requiring re-running the full code. Problem: every month a two-person analyst team would spend two days stitching up multiple CSV files (can be up to 60) via VLookup for a monthly report. This is because analysts have to look up information for each record, in which they use a template query and manually execute 50 queries with changed parameters. ...