Pandasintro.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[{"file_id":"1vPFmAVlgtcR9yippbL4_6JAVjMFVcies","timestamp":1705559660361}],"authorship_tag":"ABX9TyO5PfbV3uEYNv7MZhl8+lS9"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","source":["Pandas is a powerful Python library for data manipulation and analysis. It provides data structures like DataFrame and Series, making it easy to handle and analyze structured data. With functions for cleaning, filtering, and transforming data, Pandas is widely used in data science and analysis tasks."],"metadata":{"id":"jwG7v5kKNw96"}},{"cell_type":"markdown","source":["# Why do we use pandas\n","\n","\n","*   Data Structures(Series and Dataframe Structures)\n","*   Ease of data cleaning\n","*   Data Exploration\n","*   Handling different data types\n","\n"],"metadata":{"id":"AuAzAztOQ2Vv"}},{"cell_type":"code","source":["# run pip install pandas if you can't import pandas\n","import pandas as pd"],"metadata":{"id":"anPYknmjN_4L"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["DataFrames are two-dimensional, tabular data structures in the Pandas library. They are similar to spreadsheets or SQL tables, with rows and columns. Each column in a DataFrame can be of a different data type (e.g., integers, floats, strings), and you can perform various operations like filtering, grouping, and merging to analyze and manipulate the data efficiently. DataFrames are a key feature of Pandas, making it easier to work with and analyze structured data."],"metadata":{"id":"-WG4iWnfOa5d"}},{"cell_type":"code","source":["# creating dataframes\n","# Creating a DataFrame from a dictionary\n","data = {'Name': ['John', 'Alice', 'Bob'],\n","        'Age': [28, 24, 22],\n","        'City': ['New York', 'Los Angeles', 'Chicago']}\n","\n","df = pd.DataFrame(data)"],"metadata":{"id":"eySnSCyiOmT0"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Loading and saving data"],"metadata":{"id":"uvbv_PhYOzb4"}},{"cell_type":"code","source":["# Reading from a CSV file\n","df_csv = pd.read_csv('example.csv')\n","\n","# Saving to a CSV file\n","df.to_csv('new_dataframe.csv', index=False)"],"metadata":{"id":"2pjyX5oNO2Qu"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["In Pandas, you can inspect your data using various methods. Some common ones include:\n","\n","1. **head():** Displays the first few rows of the DataFrame.\n","   ```python\n","   df.head()\n","   ```\n","\n","2. **tail():** Shows the last few rows of the DataFrame.\n","   ```python\n","   df.tail()\n","   ```\n","\n","3. **info():** Provides information about the DataFrame, including data types and missing values.\n","   ```python\n","   df.info()\n","   ```\n","\n","4. **describe():** Generates descriptive statistics, like mean and standard deviation, for numerical columns.\n","   ```python\n","   df.describe()\n","   ```\n","\n","5. **shape:** Returns the number of rows and columns in the DataFrame.\n","   ```python\n","   df.shape\n","   ```\n","\n","These functions help you get a quick overview of your dataset and understand its structure and content."],"metadata":{"id":"WVOinR6WPHeh"}},{"cell_type":"markdown","source":["Data Selection and Indexing"],"metadata":{"id":"E6F8CrAsPYYy"}},{"cell_type":"code","source":["# Selecting columns\n","df['Name']\n","\n","# Filtering rows\n","df[df['Age'] > 25]\n","\n","# Setting a new index\n","df.set_index('Name', inplace=True)"],"metadata":{"id":"_NIQVOQGPasf"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Data Cleaning"],"metadata":{"id":"RgSp85nXPilX"}},{"cell_type":"code","source":["# Handling missing values\n","df.dropna()\n","\n","# Removing duplicates\n","df.drop_duplicates()"],"metadata":{"id":"NtQsUekWPkcu"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Data Manipulation with pandas"],"metadata":{"id":"xenU43OiPqNL"}},{"cell_type":"code","source":["# Merging DataFrames\n","merged_df = pd.merge(df1, df2, on='common_column')\n","\n","# Grouping and aggregating\n","grouped_df = df.groupby('City').mean()\n","\n","# Applying functions\n","df['Age'] = df['Age'].apply(lambda x: x + 1)"],"metadata":{"id":"_rTulOGnPtCZ"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Data Visualization with pandas"],"metadata":{"id":"KrZHDpt-P0K6"}},{"cell_type":"code","source":["# Plotting data\n","df.plot(kind='bar', x='Name', y='Age')"],"metadata":{"id":"Mf-pBK6BP2vZ"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Understanding Categorical and Numerical  data"],"metadata":{"id":"K6vkqYi9QK5C"}},{"cell_type":"markdown","source":["Pandas provides various methods to handle both categorical and numerical data:\n","\n","**Handling Numerical Data:**\n","1. **Descriptive Statistics:** Use `describe()` to get statistical summary.\n","   ```python\n","   df.describe()\n","   ```\n","\n","2. **Math Operations:** Perform mathematical operations on numerical columns.\n","   ```python\n","   df['numerical_column'].mean()\n","   ```\n","\n","**Handling Categorical Data:**\n","1. **Value Counts:** Check the distribution of categorical values.\n","   ```python\n","   df['categorical_column'].value_counts()\n","   ```\n","\n","2. **Label Encoding:** Convert categorical values to numerical labels.\n","   ```python\n","   from sklearn.preprocessing import LabelEncoder\n","   le = LabelEncoder()\n","   df['encoded_column'] = le.fit_transform(df['categorical_column'])\n","   ```\n","\n","3. **One-Hot Encoding:** Create binary columns for each category (useful for machine learning models).\n","   ```python\n","   df_encoded = pd.get_dummies(df, columns=['categorical_column'])\n","   ```\n","\n","These methods allow you to analyze, preprocess, and prepare both numerical and categorical data for various data science tasks."],"metadata":{"id":"FVv4PDdLQSUX"}},{"cell_type":"code","source":[],"metadata":{"id":"H1wL9KTxQa5g"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":["Reshaping Data"],"metadata":{"id":"EgBrc9M3QmM0"}},{"cell_type":"code","source":["# Melting DataFrames\n","df_melted = pd.melt(df, id_vars=['Name'], var_name='Attribute', value_name='Value')\n","\n","# Pivoting DataFrames\n","df_pivoted = df.pivot(index='Name', columns='Attribute', values='Value')"],"metadata":{"id":"fo_unDzwQpQV"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":[],"metadata":{"id":"nfOLqxUzQr9I"}}]}