Examples/quickstart.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parsing SEC filings with SEC Parsers\n",
    "\n",
    "Declare our imports, and create a nice helper function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sec_parsers import Filing, download_sec_filing, set_headers\n",
    "\n",
    "def print_first_n_lines(text, n):\n",
    "    lines = text.split('\\n')\n",
    "    for line in lines[:n]:\n",
    "        print(line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download Tesla's 10K 2023 Filing after setting headers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_headers(\"John Test\",'johntest@example.com')\n",
    "html = download_sec_filing('https://www.sec.gov/Archives/edgar/data/1318605/000162828024002390/tsla-20231231.htm')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a new Parser object and examine its html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Element html at 0x1c124d54380>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filing = Filing(html)\n",
    "filing.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Parse the filing and examine the new filing.xml attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Element root at 0x1c124fabc00>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "filing.parse() # parses filing\n",
    "filing.xml\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print the tree structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      "document\n",
      "|-introduction\n",
      "|-part\n",
      "|-|-item\n",
      "|-|-|-company_designated_header\n",
      "|-|-|-|-company_designated_header\n",
      "|-|-|-|-company_designated_header\n",
      "|-|-|-|-|-company_designated_header\n",
      "|-|-|-|-|-|-company_designated_header\n"
     ]
    }
   ],
   "source": [
    "print_first_n_lines(filing.get_tree(),10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Titles are more descriptive than tags, so lets look at those by printing the title tree."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Document\n",
      "|-introduction\n",
      "|-PART I\n",
      "|-|-ITEM 1. BUSINESS\n",
      "|-|-|-Overview\n",
      "|-|-|-|-Segment Information\n",
      "|-|-|-|-Our Products and Services\n",
      "|-|-|-|-|-Automotive\n",
      "|-|-|-|-|-|-Energy Generation and Storage\n",
      "|-|-|-|-|-|-|-Energy Storage Products\n",
      "|-|-|-|-|-|-|-Solar Energy Offerings\n",
      "|-|-|-|-Technology\n",
      "|-|-|-|-|-Automotive\n",
      "|-|-|-|-|-|-Battery and Powertrain\n",
      "|-|-|-|-|-|-Vehicle Control and Infotainment Software\n",
      "|-|-|-|-|-|-Self-Driving Development and Artificial Intelligence\n",
      "|-|-|-|-|-|-|-Energy Generation and Storage\n",
      "|-|-|-|-|-|-|-Energy Storage Products\n",
      "|-|-|-|-|-|-|-Solar Energy Systems\n"
     ]
    }
   ],
   "source": [
    "print_first_n_lines(filing.get_title_tree(),20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Find section from title. To return a list, use find_all_sections_from_title\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "item1a = filing.find_section_from_title('item 1a')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SEC Parsers parses titles of sections into the title attribute and removes it from the original text. You can access the text of a node using traditional xml methods, but it will be missing the titles of the node and its children.\n",
    "\n",
    "To avoid that, use get_text_from_section. include_title = True tells the function to include the sections title. By default this is set to false. Subsection titles are always returned by get_text_from_section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ITEM 1A. RISK FACTORS\n",
      "\n",
      "You should carefully consider the risks described below together with the other information set forth in this report, which could materially affect our business, financial condition and future results. The risks described below are not the only risks facing our company. Risks and uncertainties not currently known to us or that we currently deem to be immaterial also may materially adversely affect our business, financial condition and operating results.\n",
      "\n",
      "Risks Related to Our Ability to Grow Our Business\n",
      "\n",
      "We may experience delays in launching and ramping the production of our products and features, or we may be unable to control our manufacturing costs.\n",
      "We have previously experienced and may in the future experience launch and production ramp delays for new products and features. For example, we encountered unanticipated supplier issues that led to delays during the initial ramp of our first Model X and experienced challenges with a supplier and with ramping full automation for certain of our initial Model 3 manufacturing processes. In addition, we may introduce in the future new or unique manufacturing processes and design features for our products. As we expand our vehicle offerings and global footprint, there is no guarantee that we will be able to successfully and timely introduce and scale such processes or features. In particular, our future business depends in large part on increasing the production of mass-market vehicles. In order to be successful, we will need to implement, maintain and ramp efficient and cost-effective manufacturing capabilities, processes and supply chains and achieve the design tolerances, high quality and output rates we have planned at our manufacturing facilities in California, Nevada, Texas, China, Germany and any future sites such as Mexico. We will also need to hire, train and compensate skilled employees to operate these facilities. Bottlenecks and other unexpected challenges such as those we experienced in the past may arise during our production ramps, and we must address them promptly while continuing to improve manufacturing processes and reducing costs. If we are not successful in achieving these goals, we could face delays in establishing and/or sustaining our product ramps or be unable to meet our related cost and profitability targets. We have experienced, and may also experience similar future delays in launching and/or ramping production of our energy storage products and Solar Roof; new product versions or variants; new vehicles; and future features and services based on artificial intelligence. Likewise, we may encounter delays with the design, construction and regulatory or other approvals necessary to build and bring online future manufacturing facilities and products. Any delay or other complication in ramping the production of our current products or the development, manufacture, launch and production ramp of our future products, features and services, or in doing so cost-effectively and with high quality, may harm our brand, business, prospects, financial condition and operating results. \n",
      "Our suppliers may fail to deliver components according to schedules, prices, quality and volumes that are acceptable to us, or we may be unable to manage these components effectively.\n",
      "Our products contain thousands of parts purchased globally from hundreds of suppliers, including single-source direct suppliers, which exposes us to multiple potential sources of component shortages. Unexpected changes in business conditions, materials pricing, including inflation of raw material costs, labor issues, wars, trade policies, natural disasters, health epidemics such as the global COVID-19 pandemic, trade and shipping disruptions, port congestions, cyberattacks and other factors beyond our or our suppliers’ control could also affect these suppliers’ ability to deliver components to us or to remain solvent and operational. For example, a global shortage of semiconductors beginning in early 2021 has caused challenges in the manufacturing industry and impacted our supply chain and production. Additionally, if our suppliers do not accurately forecast and effectively allocate production or if they are not willing to allocate sufficient production to us, or face other challenges such as insolvency, it may reduce our access to components and require us to search for new suppliers. The unavailability of any component or supplier could result in production delays, idle manufacturing facilities, product design changes and loss of access to important technology and tools for producing and supporting our products, as well as impact our capacity expansion and our ability to fulfill our obligations under customer contracts. Moreover, significant increases in our production or product design changes by us have required and may in the future require us to procure additional components in a short amount of time. We have faced in the past, and may face suppliers who are unwilling or unable to sustainably meet our timelines or our cost, quality and volume needs, which may increase our costs or require us to replace them with other sources. Finally, as we construct new manufacturing facilities and add production lines to existing facilities, we may experience issues in correspondingly increasing the level of localized procurement at those facilities. While we believe that we will be able to secure additional or alternate sources or develop our own replacements for most of our components, there is no assurance that we will be able to do so quickly or at all. Additionally, we may be unsuccessful in our continuous efforts to negotiate with existing suppliers to obtain cost reductions and avoid unfavorable changes to terms, source less expensive suppliers for certain parts and redesign certain parts to make them less expensive to produce, especially in the case of increases in materials pricing. Any of these occurrences may harm our business, prospects, financial condition and operating results.As the scale of our vehicle production increases, we will also need to accurately forecast, purchase, warehouse and transport components at high volumes to our manufacturing facilities and servicing locations internationally. If we are unable to accurately match the timing and quantities of component purchases to our actual needs or successfully implement automation, inventory management and other systems to accommodate the increased complexity in our supply chain and parts management, we may incur unexpected production disruption, storage, transportation and write-off costs, which may harm our business and operating results.\n"
     ]
    }
   ],
   "source": [
    "item1a_text = filing.get_text_from_section(item1a,include_title=True)\n",
    "print_first_n_lines(item1a_text,10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Find sections by text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Our products contain thousands of parts purchased globally from hundreds of suppliers, including single-source direct suppliers, which exposes us to multiple potential sources of component shortages. Unexpected changes in business conditions, materials pricing, including inflation of raw material costs, labor issues, wars, trade policies, natural disasters, health epidemics such as the global COVID-19 pandemic, trade and shipping disruptions, port congestions, cyberattacks and other factors beyond our or our suppliers’ control could also affect these suppliers’ ability to deliver components to us or to remain solvent and operational. For example, a global shortage of semiconductors beginning in early 2021 has caused challenges in the manufacturing industry and impacted our supply chain and production. Additionally, if our suppliers do not accurately forecast and effectively allocate production or if they are not willing to allocate sufficient production to us, or face other challenges such as insolvency, it may reduce our access to components and require us to search for new suppliers. The unavailability of any component or supplier could result in production delays, idle manufacturing facilities, product design changes and loss of access to important technology and tools for producing and supporting our products, as well as impact our capacity expansion and our ability to fulfill our obligations under customer contracts. Moreover, significant increases in our production or product design changes by us have required and may in the future require us to procure additional components in a short amount of time. We have faced in the past, and may face suppliers who are unwilling or unable to sustainably meet our timelines or our cost, quality and volume needs, which may increase our costs or require us to replace them with other sources. Finally, as we construct new manufacturing facilities and add production lines to existing facilities, we may experience issues in correspondingly increasing the level of localized procurement at those facilities. While we believe that we will be able to secure additional or alternate sources or develop our own replacements for most of our components, there is no assurance that we will be able to do so quickly or at all. Additionally, we may be unsuccessful in our continuous efforts to negotiate with existing suppliers to obtain cost reductions and avoid unfavorable changes to terms, source less expensive suppliers for certain parts and redesign certain parts to make them less expensive to produce, especially in the case of increases in materials pricing. Any of these occurrences may harm our business, prospects, financial condition and operating results.As the scale of our vehicle production increases, we will also need to accurately forecast, purchase, warehouse and transport components at high volumes to our manufacturing facilities and servicing locations internationally. If we are unable to accurately match the timing and quantities of component purchases to our actual needs or successfully implement automation, inventory management and other systems to accommodate the increased complexity in our supply chain and parts management, we may incur unexpected production disruption, storage, transportation and write-off costs, which may harm our business and operating results.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "section = filing.find_all_sections_from_text(\"Moreover, significant increases in our production or product design changes by us have required and may in the future require us to procure additional components in a short amoun\")[0]\n",
    "section_text = filing.get_text_from_section(section,include_title=False)\n",
    "print_first_n_lines(section_text,10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Find subsections"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Element item at 0x1c123621780>,\n",
       " <Element item at 0x1c1250c72c0>,\n",
       " <Element item at 0x1c124d71640>,\n",
       " <Element item at 0x1c1250c6d40>,\n",
       " <Element item at 0x1c1250c7b00>,\n",
       " <Element item at 0x1c1250c53c0>,\n",
       " <Element item at 0x1c1250c6180>]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "section = filing.find_section_from_title('part i')\n",
    "filing.get_subsections_from_section(section)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save data to xml or csv. Note that saving to csv will be larger due to duplication. Supported encodings are utf-8 and ascii."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "filing.save_xml('tesla_10k.xml', encoding='utf-8')\n",
    "filing.save_csv('tesla_10k.csv',encoding='ascii')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize the parsing in webbrowser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "filing.visualize()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}