Foundation data Week 1

Case Study: New data perspectives

- six steps of the data analysis process: ask, prepare, process, analyze, share, and act.?

????- Ask: define what the project would look like and what would qualify as a successful result. To determine these things, they asked effective questions

????- Prepare: Build a timeline and decide how you want to relay your progress to interested parties. Also, during this step, the analysts identified what data they needed to achieve the successful result they identified in the previous step

????- Process: Since employees provided the data, it was important to make sure all employees gave their consent to participate. The data analysts also made sure employees understood how their data would be collected, stored, managed, and protected. They processed the data by cleaning it to make sure it was complete, correct, relevant, and free of errors and outliers.?

????- Analyze: data analysts discover a result/fact by analyzing

????- Share: Just as they made sure the data was carefully protected, the analysts were also careful in sharing the report

????- Act: The last stage of the process for the team of analysts was to work with leaders within their company and decide how best to implement changes and take actions based on the findings.?

????- additional resource: https://online.hbs.edu/blog/post/business-analytics-examples

Data ecosystems

- define: the various elements that interact with one another in order to produce, manage, store, organize, analyze, and share data

Data science vs data analysts

- data science: creating new ways of modelling and understanding the unknown by using raw data

- data analysts: find answers to existing questions by creating insights from data sources

Data analysis vs data analytics

- data analysis: the collection, transformation, and organization of data in order to draw conclusions, make predictions, and drive informed decision-making

- data analytics: the science of data

Data-driven decision-making

- define: Using facts to guide business strategy

subject matter expert?

- have the ability to look at the results of data analysis and identify any inconsistencies, make sense of gray areas, and eventually validate choices being made

Data and gut instinct

- Gut instinct is an intuitive understanding of something with little or no explanation.?

- Problem: decisions may be biased based on gut instinct

Data analysis life cycle

- the process of going from data to decision

Process

1. Ask: Business Challenge/Objective/Question

2. Prepare: Data generation, collection, storage, and data management

3. Process: Data cleaning/data integrity

4. Analyze: Data exploration, visualization, and analysis

5. Share: Communicating and interpreting results?

6. Act:?Putting your insights to work to solve the problem

EMC's data analysis life cycle

1. Discovery

2. Pre-processing data

3. Model planning

4. Model building

5. Communicate results

6. Operationalize

- reflects the cyclical nature of real-world projects

- the idea that is in common with the data analysis life cycle: the first phase is interested in discovering and asking questions; data has to be prepared before it can be analyzed and used; and then findings should be shared and acted on.

SAS's iterative life cycle

- can be used to produce a repeatable, reliable, and predictive result

- it includes a step after the act phase designed to help analysts evaluate their solutions and potentially return to the asking phase again

1. Ask

2. Prepare

3. Explore

4. Model

5. Implement

6. Act

7. Evaluate

Project-based data analytics life cycle?

1. Identifying the problem

2. Designing data requirements

3. Pre-processing data

4. Performing data analysis

5. Visualizing data

- More: http://pingax.com/understanding-data-analytics-project-life-cycle/

Big data analytics life cycle

- Big Data Fundamentals: Concepts, Drivers & Techniques.

1. Business case evaluation

2. Data identification

3. Data acquisition and filtering

4. Data extraction

5. Data validation and cleaning?

6. Data aggregation and representation

7. Data analysis

8. Data visualization

9. Utilization of analysis results

- More: https://www.informit.com/articles/article.aspx?p=2473128&seqNum=11&ranMID=24808

??????????????Week 2?

Analytical skill

- qualities and characteristics associated with solving problems using facts

1. Curiosity: wanting to learn something?

2. Understanding context: context is the condition in which something exists or happens

3. Having a technical mindset: involves the ability to break things down into smaller steps or pieces and work with them in an orderly and logical way

4. Data design: how you organize information

5. Data strategy: the management of the people, processes, and tools used in data analysis.

Analytical thinking

- Identifying and defining a problem and then solving it by using data in an organized, step-by-step manner

1. Visualization: the graphical representation of information (graph)

2. Strategy: help to see what the data analysts want to achieve with the data and how they can get there

3. Problem-orientation: to identify, describe and solve problems

4. Correlation: correlation does not equal causation

5. Big-picture and detail-oriented thinking: being able to see the big picture as well as detail

?Questions data analysts may ask

- What is the root cause of a problem?

-Root cause: the reason why a problem occurs

-Ask five “why”

- Where are the gaps in our process?

- Gap analysis: a method for examining and evaluating how a process works currently in order to get where you want to be in the future

- What did we not consider before?

Quartile

- A quartile divides data points into four equal parts

Nonprofits

- Organizations dedicated to advancing a social cause or advocating for a particular effort

????????????Week 3

The life cycle of data

1. Plan: business decides what kind of data it needs, how it will be managed throughout its life cycle, who will be responsible for it, and the optimal outcomes

2. Capture: data is collected from a variety of different sources and brought into the organization (outside resources or get from a company’s own document which is usually stored inside a database, and a database is a collection of data stored in a computer system)

3. Manage: how we care for our data, how and where it's stored, the tools used to keep it safe and secure, and the actions taken to make sure that it's maintained properly.

4. Analyze: the data is used to solve problems, make great decisions, and support business goals

5. Archive: Archiving means storing data in a place where it's still available, but may not be used again

6. Destroy: use secure data erasure software to destroy it, this is important for protecting a company's private information, as well as private data about its customers

U.S. Fish and Wildlife Service

The U.S. Fish and Wildlife Service uses the following data life cycle:

1. Plan

2. Acquire

3. Maintain

4. Access?

5. Evaluate

6. Archive

- More: https://www.fws.gov/data/life-cycle

The U.S. Geological Survey (USGS)

The USGS uses the data life cycle below:

1. Plan

2. Acquire

3. Process

4. Analyze

5. Preserve

6. Publish/Share

Several cross-cutting or overarching activities are also performed during each stage of their life cycle:

? Describe (metadata and documentation)

? Manage Quality

? Backup and Secure

- More: https://www.usgs.gov/products/data-and-tools/data-management/data-lifecycle

Other data life cycle definitions: Financial institutions (https://sfmagazine.com/post-entry/july-2018-the-data-life-cycle/) and Harvard Business School (https://online.hbs.edu/blog/post/data-life-cycle)

Stakeholder

- people who have invested time and resources into a project and are interested in the outcome

Six phases of data analysis

1. Ask: Look at the current state and identify how it’s different from the ideal state (Another important part of the asking phase is understanding stakeholder expectations. The first step here is to determine who the stakeholders are. That may include your manager, an executive sponsor, or your sales partners.)

2. Prepare: where data analysts collect and store data, they'll use for the upcoming analysis process.

3. Process: find and eliminate any errors and inaccuracies that can get in the way of results. This usually means cleaning data, transforming it into a more useful format, combining two or more datasets to make information more complete and removing outliers, which are any data points that could skew the information.?

4. Analyze: involves using tools (i.e., spreadsheets and structured query language) to transform and organize that information so that you can draw useful conclusions, make predictions, and drive informed decision-making

5. Share: visualization is important?

6. Act

Tools

- Spreadsheets:?

-Microsoft Excel and Google Sheets. The spreadsheet is a digital worksheet. It stores, organizes and sorts data.

-useful features: formulas (a set of instructions that performs a specific?

calculation using the data in a spreadsheet) and functions (a preset command that automatically performs a specific process or task using the data in a spreadsheet)

- Query languages for databases: a computer programming language that allows you to retrieve and manipulate data from a database, such as SQL which is a language that lets data analysts communicate with a database. Allow analysts to select, create, add, or download data from a database for analysis

- Visualization tools: such as Tableau (simple drag-and-drop feature lets users create??interactive graphs in dashboards and worksheets) and Looker (communicates directly with a database, allowing you to connect your data right to the visual tool you choose)

?????????????????Week 4

- Attribute: a characteristic or quality of data used to label a column in a table

- Row = observation: All of the attributes for something contained in a row of a data table

- Formula: a set of instructions that performs a specific action using the data in a spreadsheet

SQL

- Store

- Organize

- Analyze

- Larger scale

The basic structure of a SQL query

- Select [choose the column(s) you want] #2

- From [from the appropriate table, choose the tables where the columns you want are located] #1

- Where [a certain condition is met, filter the certain information] #3

- Select: use an asterisk (*) to select all of the data from the table

Query

- A request for data or information from a databas

月?19:26:48

- Syntax: the predetermined structure of a language that includes all required words, symbols, and punctuation, as well as their proper placement

- unlike the SELECT command which uses a comma to separate fields/variables/parameters, the WHERE command uses the AND statement to connect conditions.

- Semicolon: statement terminator

- Percent sign (%): used as a wildcard to match one or more characters (some databases use asterisk)

- SELECT*：selecting all of the columns in the table?

- Comments: text placed between certain characters, /* and */, or after two dashes (--)

- Aliases (done with AS): can make it easier by assigning a new name or alias to the column to make them easier to work with

- <> = does not equal

Platform for visualizations

- Tableau

- RStudio

???????????????Week 5

Issue

- A topic or subject to investigate

Question

- Designed to discover information

Problem

- An obstacle or complication that needs to be worked out

Business task

- The question or problem data analysis answers for a business (analyze weather data from the last decade to identify predictable patterns)

Fairness

- Ensuring that your analysis doesn’t create or reinforce bias