AK.0.3/A.K.03
2
0
Fork 0
mirror of https://gitlab.ewi.tudelft.nl/ee2l1/2025-2026/A.K.03.git synced 2025-12-12 13:50:57 +01:00
Code Issues Packages Projects Releases Wiki Activity

initial commit

Browse Source
This commit is contained in:
Mehrdad Mihankhah 2025-11-16 15:37:13 +01:00
commit 12cabf65ea
98 changed files with 7459 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
0_Table_of_Contents.ipynb Normal file
View File

@ -0,0 +1,47 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](pictures/header.png)\n",
"# Table of Contents\n",
"\n",
"\n",
"Chapter 1: [Introduction](1_Introduction.ipynb)\n",
"\n",
"Chapter 2: [Module 0, Before We Start](2_Before_Re_Start.ipynb)\n",
"\n",
"Chapter 3: [Module 1, Connecting with and Controlling KITT](3_Module_1.ipynb)\n",
"\n",
"Chapter 4: [Module 2, Reading KITT Sensor Data](4_Module_2.ipynb)\n",
"\n",
"Chapter 5: [Module 3, Locating KITT Using Audio Communication ](5_Module_3.ipynb)\n",
"\n",
"Chapter 6: [ Module 4, Car Model](6_Module_4.ipynb)\n",
"\n",
"Chapter 7: [ Mid-term report](7_Midterm_Report.ipynb)\n",
"\n",
"Chapter 8: [Module 5 - Navigating KITT and Final Challenges](8_Module_5.ipynb) \n",
"\n",
"Appendix A: [Programming the Audio Beacon](appendix/Appendix_A.md)\n",
"\n",
"Appendix B: [Serial Communication with Windows](appendix/Appendix_B.md)\n",
"\n",
"Appendix C: [TDOA Localization Algorithm](appendix/Appendix_C.ipynb)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

308
1_Introduction.ipynb Normal file
View File

@ -0,0 +1,308 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](pictures/header.png)\n",
"\n",
"[Table of Contents](0_Table_of_Contents.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Chapter 1: Introduction\n",
"\n",
"**Contents:**\n",
"* [Project description](#project_description)\n",
"* [Teamwork Division](#teamwork)\n",
"* [Schedule and Deadlines](#schedule-and-deadlines)\n",
"* [Assessment and Report](#assessment-and-reports)\n",
"* [Facilities](#facilities)\n",
"* [Rules and Regulations](#rules-and-regulations)\n",
"* [Educational Objectives](#educational-objectives)\n",
"\n",
"\n",
"Welcome to the EE2L1 IP3 project, “KITT: Autonomous Driving Challenge.” In this 5-EC course project, your team will develop and program an autonomous electric toy car, \"KITT,\" capable of navigating independently from a designated start point to a target location in a field, with you only needing to press the start button.\n",
"KITT is equipped with distance sensors and an audio beacon and communicates with a base station (your computer) via Bluetooth, sending data, receiving commands and control inputs. The field is fitted with five microphones that capture the beacon's signal to facilitate precise localization of the car. \n",
"\n",
"Like in the previous integrated projects, the objective of IP3 is to apply and integrate the EE knowledge you have acquired so far (mostly signals & systems), while we also look forward to telecommunication, system modeling and control. This project heavly relies on Python programming, and you will have to design and implement a system that integrates these aspects.\n",
"\n",
"This manual starts with a description of the overall project. This complex task is split into modules, each contributing to the overall goal. "
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": [
"#project_description_tag"
]
},
"source": [
"## Project Description\n",
"\n",
"The overall goal of the project is to take a standard toy car with added functionality so that it can be remotely operated, can detect obstacles in front of it, and can communicate with a base station (your PC) via Bluetooth. The base station performs calculations such as location estimation (using beacon audio signals recorded by 5 microphones in the field), trajectory tracking, and collision avoidance (optional) as shown in the figure below. You will have to develop these functionalities. You will also have to document the designs in your reports, describe the design choices, and evaluate the test results. Lastly, you will have to present and defend your reports and designs.\n",
"\n",
"<img src=\"pictures/intro_setup.jpg\" alt=\"Illustration of the field and KITT\" width=\"600\" />\n",
"\n",
"To complete this complex project, you’ll need to build and integrate several key modules that enable KITT’s remote connectivity, sensor data processing, localization, and real-time control:\n",
"\n",
"0. Before we can start doing anything, all team members need to set up their programming environment in Python. If you have not done so before, look at chapter 2 and follow the instructions. Besides Python, we also need an Integrated Development Environment (IDE), and some required packages for the project.\n",
"\n",
"1. *Connecting to and controlling KITT*: this module consists of setting up an RF connection to KITT and writing a program that allows you to control KITT and drive it around. The connection is provided by off-the-shelf Bluetooth units, controlled by a microcontroller on the car and an interface on the base station PC. The microcontroller can also generate motor control signals using basic commands from the base station PC.\n",
"\n",
"2. *Reading KITT sensor data* and detecting objects: this module consists of reading and using the data from the anti-collision sensors and the microphones along the field. These distance sensors are a pair of off-the-shelf ultrasonic transmitters/receivers that can detect objects in front of them.\n",
"\n",
" - The maximal range of the detection is about $6$ m.\n",
" - The system is mounted on the car.\n",
" - The system is controlled by the onboard microcontroller and read out via the RF link.\n",
"\n",
"3. *Localization using audio communication* is done by transmitting a repetitive beacon signal from the car. This is an audio signal, and it may be corrupted by interfering signals emitted by other cars present in the field (depending on whether you choose to do this challenge), and by surrounding noise.\n",
"\n",
" - The audio signals from all cars are captured by multiple microphones placed along the field. The analog microphone signals are sampled and made available to you.\n",
" - There are 5 microphones around the field, of which the location is known.\n",
" - The estimation of the location is done in Python by using the microphone data and the TDOA algorithm you developed in the Assignment in Week 1. \n",
" - The location updates should be frequent and accurate enough for the trajectory tracking to function. (Minimal numbers are at least one update per $2$ seconds, with an accuracy better than 30cm.)\n",
"\n",
"4. *Car model: Control and Trajectory tracking* is based on a sufficiently accurate system model of the car, both regarding velocity and steering behavior.\n",
"\n",
" - The state of KITT consists of position, velocity, and orientation (4 parameters).\n",
" - The velocity model is based on differential equations that take the speed settings into account.\n",
" - The steering behavior is based on modeling the trajectory resulting from a steering command.\n",
" - The model of the car should allow you to predict the next state of the car, given the current state and the control input.\n",
"\n",
"5. *System Integration: Navigating KITT and Final Challenges* brings your previously done work together. KITT's starting location and orientation are known, and from this you will have to route KITT to certain points in the field.\n",
"\n",
" - The system you make must be closed loop, control input is from your localization algorithm that uses an audio beacon on top of the car, additionally you may have to use the data from the anti-collision sensors. It is important to note that \"open loop\" solutions will not be accepted. These types of solutions blindly rely on the model and do not take the sensor data and localization estimates into account. \n",
" - The control actions are computed in Python on your base station PC. \n",
" - After the start of the challenge, you are not allowed to touch the base station PC.\n",
" - You have to reach the target within 5 minutes. Ideally, you drive to the target without stopping, and measure your position while driving. In this case, there are many real-time aspects to take into account. In a basic (permitted) solution, you pause the car frequently to measure its position, and then drive another ~50 cm. \n",
" - The main considerations in the design are the proper managing of uncertainties in localization, state updates, and real-time aspects.\n",
"\n",
"Each of these functionalities is specified in its own module.\n",
"\n",
"An overview of the entire system is shown in the system overview figure below. In the control loop, the car model predicts the car position (and its velocity and orientation), and this is used to assist in tracking the state without independent position estimate from the microphones. This block is not absolutely necessary; however, without it, you probably need a pretty good localization system. \n",
"\n",
"The total duration of the project is 8 weeks. Each week, you have two lab sessions; outside of the lab, you must prepare the lab assignments, homework, and report writing. The project is finished with the Final Challenge (demo session, week 9) and the Presentation and Defense (committee interview, week 10).\n",
"\n",
"<img src=\"pictures/projectBD.png\" alt=\"An example architecture of the KITT control system\" width=\"600\" />"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Teamwork\n",
"\n",
"Effective collaboration is essential for the successful completion of this project. Since your team comprises four or, in some cases, five students, we recommend dividing the work into two parallel work packages. Before the midterm, consider forming two subgroups: one group should focus on Modules 1 and 2, while the other works on Modules 3 and 4. Note that both subgroups should read chapter 1 and 2 before starting with their own modules.\n",
"After completing the midterm report, the team will need to integrate all modules, allowing KITT to navigate the field from point A to point B. \n",
"\n",
"### Subgroup 1. Communicating with KITT \n",
"\n",
"Two team members can focus on the communication aspect. This involves establishing a reliable and efficient communication link between the computer (base station) and KITT. Tasks include:\n",
"\n",
"- Implementing communication over Bluetooth.\n",
"- Developing functions to send commands to KITT.\n",
"- Creating modules to read sensor data from KITT.\n",
"- Characterizing the sensors and motions of KITT.\n",
"- Making recordings using the microphones (to be used by subgroup 2).\n",
"\n",
"Communication with KITT is discussed in Modules 1 and 2.\n",
"\n",
"### Subgroup 2. Localization algorithm development\n",
"\n",
"The other two team members can focus on the localization algorithm. This involves processing audio data from the microphones to estimate the location of KITT. Tasks include:\n",
"\n",
"- Developing a channel estimation algorithm for received audio signals. (A basis for this has been designed during the initial Assignments)\n",
"- Implementing Time Difference of Arrival (TDOA) calculations.\n",
"- Using TDOA data locate KITT\n",
"- Handling data synchronization and ensuring accuracy in localization.\n",
"\n",
"This is further explained in Module 3.\n",
"\n",
"### Subgroup (whoever is first)\n",
" - Developing a car model.\n",
"\n",
" This is further explained in Module 4. The goal is to take control inputs and predict the car position ~1 second ahead of time.\n",
"\n",
"### Collaboration\n",
"\n",
"You maximize efficiency and expertise by assigning dedicated team members to each area in parallel. Remember that effective communication within the team is critical. Regular updates, meetings, and shared documentation will help keep everyone on the same page and facilitate successfully integrating the individual components into the autonomous driving system. To better understand what the other group is working on, it is advisable to read the manual on the other modules.\n",
"\n",
"You will find that working in parallel is only possible if you define clear interfaces: e.g., route planning is only possible if it is clear how accurate the localization algorithm works. In turn, the localization algorithm must know time stamps to know how 'old' the microphone data is, and produce a time stamp telling how old the location fix is. As a result, despite each module functioning well independently, integration will present additional challenges. Therefore, rather than spending excessive time refining individual modules, prioritize early integration (even earlier than the midterm report deadline) to address any system-wide issues. \n",
"\n",
"### Kickoff: Getting to know your team\n",
"\n",
"To form a team with new people is not always easy. Let’s start with a simple task: filling out and submitting a document together. Please look on Brightspace for the document *ip3_kickoff.docx* (under Content - Week 2 Kickoff). The document asks questions about how you want to collaborate as a team. Submit the document in your Brightspace assignment folder.\n",
"\n",
"You should find ways to brainstorm as a team, find possible approaches to a problem (analyze the problem first!), and keep everyone busy, involved, and synchronized.\n",
"\n",
"One thing to decide is to form two sub-groups: one that will work on localization (Module 3 and 4) and the other to work on the car interface (Modules 1 and 2). After these modules are finished, the mid-term report must be written according to the guidelines in the chapter 'Midterm'.\n",
"\n",
"To further improve on team-working skills, in Week 2 you also will attend an ITAV workshop on _Scrum_. This is a technique originally developed for ICT project management. Rather than a very precise product specification and subsequent lengthy development cycle where the end result might not be what the client wanted, the idea is to have a more flexible design cycle where the product is iteratively refined. At the same time, team members are more involved and carry more responsibility. Thus, Scrum appears to be quite suitable for the way you work in IP3."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Schedule and Deadlines\n",
"\n",
"IP3 has 16 scheduled lab sessions (distributed over 8 weeks). Outside of these sessions, you will have time to prepare and work on the report. In week 9, there is a common session for the final challenge, and in week 10, you will be called for a final presentation followed by a discussion. The total lab time is 56 hours, and preparation/homework time is budgeted at 84 hours, for a total study load of 5 EC.\n",
"\n",
"The table below shows a generic schedule. For the specific dates this year, look in Brightspace! The planning regarding the completion of the modules is flexible, but the deadlines are firm.\n",
"\n",
"Many groups compete for time on the test fields: 4 to 5 groups share the same field. You must reserve time slots on one of these systems using a planner in Brightspace. Come prepared with a measurement/test plan!\n",
"\n",
"| Date | Description |\n",
"|---------|--------------------------------------------------------------------------------------------------|\n",
"| Week 1 | Assignments (audio channel estimation) |\n",
"| Week 2 | Kick-off + getting your IDE setup + ITAV Scrum + Start on modules 1, 2, 3 |\n",
"| Week 3 | Continue modules 1, 2, 3, start on Module 4 |\n",
"| Week 4 | Complete modules 1, 2, 3 |\n",
"| Week 5 | Complete module 4 + ITAV Ethics + **Midterm report deadline** |\n",
"| Week 6 | Module 5 + System integration, prepare for challenge A |\n",
"| Week 7 | Testing challenge A |\n",
"| Week 8 | Complete challenge A, start on challenges B and C; final report writing |\n",
"| Week 9 | **Final challenge + Final report deadline** |\n",
"| Week 10 | Presentation and discussion |"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Assessment and reports\n",
"\n",
"### Mid-term assessment and report\n",
"\n",
"As soon as you are ready, please demonstrate the functionality and performance of your communication, localization and car model scripts to your TA. After this, document your work in your mid-term report, detailing your approach, implementation, and test results for Modules 1--4.\n",
"\n",
"#### Modules 1, 2: Communication script assessment\n",
"\n",
"Using your communication script, you are asked to demonstrate that you can control your car using the WASD or arrow keys on the keyboard. You should be able to transmit the beacon signal and record it using the 5 microphones. The detailed requirements are outlined in the chapters on Modules 1 and 2.\n",
"\n",
"#### Module 3: Localization script assessment\n",
"\n",
"Using your localization script, you are asked to demonstrate that you can locate the car from recordings that are given to you. (Integration with your own car is not yet necessary.) Refer to the specifications outlined in the chapter on Module 3 for detailed requirements.\n",
"\n",
"#### Module 4: Car model assessment\n",
"\n",
"In Module 4, you are asked to write a basic car model that captures the dynamics of KITT: its response to driving and steering commands. You do this based on measurements of its behavior, along with simple equations of motion. The goal of this model is to be able to predict the location of KITT after a few seconds of driving. You can also use this model to test your control algorithms later.\n",
"\n",
"You demonstrate your car model by predicting the position of KITT after a short series of commands, and by comparing this to the actual outcome.\n",
"\n",
"#### Mid-term report\n",
"\n",
"You submit a report of approximately 15 pages (plus cover page and appendix) documenting your work until now.\n",
"More details on the structure and contents of this report are in the chapter on the Mid-term report.\n",
"\n",
"### Final Challenge and Report\n",
"\n",
"#### Final Challenge\n",
"\n",
"To conclude the project, your design is tested during the final challenge. The tests will be described briefly here and in more detail in the System Integration chapter.\n",
"\n",
"The final challenge starts at a known location and orientation. At a start signal, you have to drive autonomously to a target. If you complete the task successfully, you may take on more complex tasks, including additional waypoints, obstacles, and other cars. It is required to reach the target with a certain minimal accuracy. Bonus points are obtained by the fastest team for each challenge. During the race, it is not allowed to touch the car.\n",
"\n",
"Speed is not the main requirement in this challenge. You may reach the target faster by driving faster, but navigating becomes more complex, location updates must be computed more quickly, and you have less time to avoid obstacles.\n",
"\n",
"#### Final report\n",
"\n",
"The project outcome is documented in a final report. The report must follow a structured approach, which you have learned in previous projects; the midterm report is a part of it. Use a to-the-point, concise, but complete reporting style. Provide an appendix with all Python code. More details on this report are in the System Integration chapter. Your report is submitted in the corresponding submission folder in Brightspace.\n",
"\n",
"The report without an appendix should be about 30 pages. Within these pages, you must document your design choices, explain your control systems, and report the deliverables of all modules, how you combined these modules, and what problems you ran into and solved them. The focus is on your findings and measurement results and the corresponding conclusions. You are also judged regarding project skills such as planning and teamwork.\n",
"\n",
"#### Final presentation and discussion\n",
"\n",
"In week 10 (consult Brightspace for the exact date), you present and defend your final report before an examination committee. The examiners will ask questions about your design choices and aspects of teamwork. This will be part of your grade.\n",
"\n",
"The presentation lasts at most 10 min. This is rather short: focus on the highlights and special features of the design, and mention the work breakdown and distribution of tasks to team members.\n",
"\n",
"While each team member may not have been directly involved in every aspect of the project, through open communication and collaboration, all team members are expected to have an operating understanding of all parts of the project. During the discussion, detailed questions will be asked to the group. The team member with the best subject knowledge is encouraged to answer. However, less complex questions can be asked to anyone to test their participation in the project. \n",
"\n",
"The examination will last about 30 min. After the examination, you will be asked to fill in a peer review form. Individual grades are differentiated depending on staff observations and the outcome of the peer review.\n",
"\n",
"### Grading\n",
"\n",
"Your grade depends on the following:\n",
"\n",
"- Mid-term report (30\\%);\n",
"- Performance during the final challenge (20\\%);\n",
"- Final report (35\\%);\n",
"- Oral presentation and defense (15\\%).\n",
"\n",
"\n",
"Teamwork is important. Your individual grade may differ depending on staff observations and peer reviews. There is a penalty for submitting the report late.\n",
"\n",
"If your final grade is insufficient, you may have a chance to improve your grade by improving your report.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Facilities\n",
"\n",
"The project is carried out at the Tellegen Hall facilities in A and B teams. Each team has 2 sessions per week. The hall has three testing areas, each shared by 4 teams. We publish a schedule for this on Brightspace. Your testing time is limited, so plan ahead and use it wisely.\n",
"\n",
"The following support is available:\n",
"\n",
"- *Student assistants*; student assistants are your primary help. Each assistant supports up to four teams. Assistants also check attendance and progress.\n",
"- *Instructors*; Practicum coordinators are available at each lab session. The coordinators also grade your reports.\n",
"- *Technical support*; for questions about hardware and implementation issues, you can contact the student assistants and/or the technicians at the facilities.\n",
"\n",
"Visit the Brightspace page of the course for support and contact information.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Rules and Regulations\n",
"\n",
"In addition to the rules and regulations of the EEE group on the use of the Tellegen Hall, the following rules and regulations are applicable:\n",
"\n",
"- **You are expected to be present during your scheduled lab sessions.** If you cannot attend for a good reason, contact your assigned TA *before* the lab session. Absence more than two times will not be allowed; you will be removed from the practicum. \n",
"- Preparation for labdays is *mandatory*. This project has an intense pace and limited time; use it wisely. The scheduled homework time is needed.\n",
"- You may not work alone in the lab. A student assistant or staff member must be present.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Educational Objectives\n",
"\n",
"General learning objective: To integrate different technical areas related to electronic systems, signal processing and control. The educational objectives are:\n",
"\n",
"- Increased skills in building and testing electronic systems. Software skills mainly consist of advanced Python programming.\n",
"- Increased skills in measurement techniques, e.g., wireless channel measurements.\n",
"- Application of course material from various courses: linear algebra, signal transformations, digital signal processing. The project also illustrates concepts from telecommunication and control systems which are taught in subsequent courses (EE2T1, EE2S2).\n",
"- Increased academic skills related to project management: **managing an open and complex assignment, planning,** acquiring background literature, **working in teams** (distributing tasks among team members, communicating within and among subgroups), **reporting**, oral presentation.\n",
"\n",
"The main difficulties students have with this project are the complexity of the overall system and planning. You will split up your team into two sub-groups. Each delivers Python code that later has to be integrated and function together. Nobody on the team has complete, detailed knowledge of the entire design. If you don’t manage this, then you will fail. Essential parts to make this work are:\n",
"\n",
"- *System Engineering:* At an early stage, agree on an overall structure for your design, partition it into modules, and decide on specifications for the modules. Find the essential aspects that need to be done right. For this system, uncertainty in time and position plays an important role. For example, the car moves while the localization is being computed.\n",
"\n",
"- *Testing:* Each module should have a clearly defined specification, and the Python code should be tested and verified against the specifications before it is integrated into the overall system. You cannot debug the overall system if you are unsure about its parts!\n",
"\n",
"- *Planning:* You will probably find that there is insufficient time in the testing areas: it is available to your team only about 25\\% of the time, and with two sub-groups, you will also compete within your team. Thus, you must plan well on what you want to measure and test during your time slot in the testing area.\n",
"\n",
"- *Modeling:* To avoid waiting for precious testing time, it is advisable to create a (simple or more advanced) software *car model*, such that you are able to test your algorithms on the model. This also allows you to verify in detail the performance of your algorithms. To help you get started, a template framework for such a model will be provided."
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

514
2_Module_0.ipynb Normal file
View File

@ -0,0 +1,514 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](pictures/header.png)\n",
"[Table of Contents](0_Table_of_Contents.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Chapter 2: Module 0 - Before We Start\n",
"\n",
"**Contents:**\n",
"* [Initial Software Installations](#initial-software-installation)\n",
"* [Object Oriented Programming](#object-oriented-programming-in-ip3)\n",
"* [Programming in Python](#programming-in-python)\n",
"* [The Car Simulator](#the-car-simulator)\n",
"\n",
"Before beginning the project, this chapter provids guidelines on installing the required software, an introduction to Object-Oriented Programming (OOP), quick tips on Python programming, and an overview of the car simulator used in the modules. Carefully read through this introduction and complete the setup preparation before starting the modules."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initial software Installation\n",
"Before we can start doing anything, you need to set up your programming environment in Python. We have provided a step by step guide that walks you through installing Python, Visual Studio Code, and all the necessary dependencies for your project. You can find them here:\n",
"\n",
"1. [Installation Mac](appendix/0_Installation_Mac.ipynb)\n",
"2. [Installation Windows](appendix/0_Installation_Windows.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Object-Oriented Programming in IP3\n",
"\n",
"Throughout the project, you will write over 20 functions and a couple hundred lines of code. As you will find, there is exponential difficulty with the increase in project size. Writing many functions is part of the assignment, but the challenge is testing these functions and getting them to work together. You must communicate with the car and interpret the data received, accurately find the car's location, plan how to drive to the final destination, generate steering commands, and adjust the plan while you drive and discover objects. \n",
"\n",
"This tutorial introduces object-oriented programming (OOP) concepts in Python, which is a programming method that provides flexibility. It is highly recommended that you learn how to use OOP, which will be useful throughout this project and future projects. The modules will provide code snippets, assuming you understand basic OOP concepts to enhance the functionality of the KITT car project.\n",
"\n",
"### Class and object\n",
"In OOP, a class is a template or a set of instructions for creating objects. On the other hand, an object is a specific instance or realization of that template. To illustrate, let's make a blueprint for KITT using a class."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class KITT:\n",
" def __init__(self, model, color):\n",
" self.model = model\n",
" self.color = color\n",
" self.is_engine_on = False\n",
"\n",
" def start_engine(self):\n",
" self.is_engine_on = True\n",
" print(f\"{self.model}'s engine started.\")\n",
"\n",
" def stop_engine(self):\n",
" self.is_engine_on = False\n",
" print(f\"{self.model}'s engine stopped.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Attributes:** These are characteristics or properties that describe the state of an object. In the real world, consider attributes as the features defining an object. In the class definition, attributes are represented by variables. In the example, we have defined a class 'KITT' with attributes 'model', 'color', and 'is_engine_on'. Which are just some characteristic of KITT we could define.\n",
"\n",
"**Methods:** These are functions that define the behavior of an object. They represent the actions that an object can perform. Methods are defined within the class and are used to manipulate the object's state (attributes) or perform some action associated with the object. Continuing with the car example, the methods 'start_engine' and 'stop_engine' control the car's engine.\n",
"\n",
"**Self:** Within the class definition, `self' is used to refer to the object. \n",
"\n",
"**The __init__ method** is a special method called the constructor. It is automatically called when a new object is created. In this method, we initialize the *model*, *color*, and *is_engine_on* variables to the values passed.\n",
"\n",
"We can now make some instances of the class KITT. We call these objects."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if __name__ == \"__main__\":\n",
" car1 = KITT(\"TRX4\", \"black\") # Make the first instance of KITT\n",
" car2 = KITT(\"Rustler\", \"red\") # Make the second instance of KITT\n",
"\n",
" car2.color = \"blue\" # Change the color of car2\n",
" \n",
" car1.start_engine() # Start the engine of car1 \n",
" print(car1.is_engine_on) # Output: \"True\"\n",
" print(car1.model) # Output: \"TRX4\"\n",
" print(car1.color) # Output: \"black\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, two instances of KITT are made. They are made from the same KITT class (template) but are a different model and color. These are stored as attributes to the instance (also called object). It is possible to change an attribute of an object after it has been made. In this example, the color of the second car is changed to blue. The state of the engine is also stored as an attribute. It is defaulted to False (the engine is off). Now, the engine of car1 is started. When checked, car1 outputs that the engine is now set to on. \n",
"\n",
"### Method vs Function\n",
"\n",
"In Python, both methods and functions are blocks of code that can be called upon to perform specific tasks. However, there are fundamental differences between the two.\n",
"\n",
"#### Function:\n",
"\n",
"A function is a block of code that is defined outside of a class and can be called independently. It takes input arguments (if any), performs some operations, and optionally returns a result. Functions in Python are versatile and can be reused across different parts of a program.\n",
"\n",
"#### Method:\n",
"\n",
"A method, on the other hand, is a function that is associated with an object. It is defined within a class and operates on the data associated with the class (attributes). Methods are accessed through instances of the class (objects) and can modify the state of the object. The first argument of a method is always the special variable 'self', which refers to the instance of the class."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class KITT:\n",
" def __init__(self, model, color):\n",
" self.model = model\n",
" self.color = color\n",
" self.is_engine_on = False\n",
"\n",
" def start_engine(self):\n",
" self.is_engine_on = True\n",
" print(f\"{self.model}'s engine started.\")\n",
"\n",
" def stop_engine(self):\n",
" self.is_engine_on = False\n",
" print(f\"{self.model}'s engine stopped.\")\n",
"\n",
"def drive():\n",
" print(\"KITT is driving.\")\n",
"\n",
"if __name__ == \"__main__\":\n",
" car = KITT(\"TRX4\", \"black\") # Create an instance of KITT\n",
" car.start_engine() # Call the start_engine method\n",
" drive() # Call the drive function"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this example, 'start_engine' and 'stop_engine' are methods because they are defined within the KITT class and operate on the KITT object's state. On the other hand, 'drive' is a function defined outside of the class and can be called independently.\n",
"\n",
"### Hidden and Private variables\n",
"\n",
"In object-oriented programming, there are concepts of encapsulation and data hiding, which allow for better control over the accessibility of attributes and methods within a class. This helps in preventing accidental modification of data and ensures the integrity of the class.\n",
"\n",
"#### Hidden variables\n",
"\n",
"In Python, variables and methods can be hidden from the outside world using a single underscore (_) prefix. Although they can still be accessed, it indicates to other programmers that these elements are intended for internal use and should be treated as such."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class KITT:\n",
" def __init__(self, model, color):\n",
" self.model = model\n",
" self.color = color\n",
" self._is_engine_on = False # Hidden variable\n",
"\n",
" def start_engine(self):\n",
" self._is_engine_on = True\n",
" print(f\"{self.model}'s engine started.\")\n",
"\n",
" def stop_engine(self):\n",
" self._is_engine_on = False\n",
" print(f\"{self.model}'s engine stopped.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this modified version of the KITT class, the variable *_is_engine_on* is prefixed with a single underscore. This indicates that it's a hidden variable. While it can still be accessed from outside the class, the underscore serves as a convention to signal that it's intended for internal use.\n",
"\n",
"#### Private variables\n",
"\n",
"Python also supports the concept of private variables, which are variables that cannot be accessed or modified from outside the class. They are denoted by a double underscore (__) prefix."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class KITT:\n",
" def __init__(self, model, color):\n",
" self.model = model\n",
" self.color = color\n",
" self.__is_locked = True # Private variable\n",
"\n",
" def lock_car(self):\n",
" self.__is_locked = True\n",
" print(f\"{self.model} is locked.\")\n",
"\n",
" def unlock_car(self):\n",
" self.__is_locked = False\n",
" print(f\"{self.model} is unlocked.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this version, the variable *__is_locked* is prefixed with a double underscore, making it a private variable. Private variables cannot be accessed directly from outside the class. Attempting to do so will result in an AttributeError. Special methods should be made to modify these variables called getters and setters.\n",
"\n",
"### Getters and Setters\n",
"\n",
"In object-oriented programming, getters and setters are methods used to access and modify the private or hidden variables of a class, respectively. They provide controlled access to the class's attributes, allowing for validation and encapsulation of data.\n",
"\n",
"#### Getters:\n",
"\n",
"Getters are methods used to retrieve the values of private or hidden variables. They provide a way to access the state of an object without directly exposing its attributes."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class KITT:\n",
" def __init__(self, model, color):\n",
" self.model = model\n",
" self.color = color\n",
" self.__is_locked = True\n",
"\n",
" # Getter for is_locked\n",
" def is_locked(self):\n",
" return self.__is_locked"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this modified KITT class, a getter method *is_locked()* is added to retrieve the value of the private variable *__is_locked*.\n",
"\n",
"#### Setters:\n",
"\n",
"Setters are methods used to modify the values of private or hidden variables. They provide a way to update the state of an object while enforcing validation rules or constraints. For example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"class KITT:\n",
" def __init__(self, model, color):\n",
" self.model = model\n",
" self.color = color\n",
" self.__is_locked = True\n",
"\n",
" # Getter for is_locked\n",
" def is_locked(self):\n",
" return self.__is_locked\n",
"\n",
" # Setter for is_locked\n",
" def set_lock_status(self):\n",
" if self.__is_locked:\n",
" self.__is_locked = False\n",
" print(f\"{self.model} is unlocked.\")\n",
" else:\n",
" self.__is_locked = True\n",
" print(f\"{self.model} is locked.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this updated KITT class, a setter method *set_lock_status()* is added to modify the value of the private variable *__is_locked*. This setter will automatically switch the locked state of the car. If it was unlocked, it locks the car, and vice verso. \n",
"\n",
"**Example:**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if __name__ == \"__main__\":\n",
" car = KITT(\"TRX4\", \"black\") # Create an instance of KITT\n",
"\n",
" # Using getter to check if the car is locked\n",
" print(car.is_locked()) # Output: True\n",
"\n",
" # Using setter to unlock the car\n",
" car.set_lock_status() # Output: \"TRX4 is unlocked.\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this example, first the getter method *is_locked()* is called to check if the car is locked. Then, the setter method *set_lock_status()* is called to change the lock status, because the car when initiated is locked, it is now unlocked."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Programming in Python\n",
"\n",
"### Hints on programming\n",
"\n",
"The above tutorial briefly introduced OOP in Python and demonstrated its application in the KITT car project. However, some of these concepts can be abstract when first introduced. Therefore, you should look for more resources as you experiment with OOP.\n",
"\n",
"- When writing the code to implement functionalities required for this project, partition the code into separate functions and always include a header block with a function. In this header block, you should briefly describe the function, the required inputs, and what the output will do. Including a changelog with author names and dates is also good practice.\n",
"- Use meaningful variable names and write many comments so that others can understand what the code is doing.\n",
"- Define variables for constants such as $F_s$ rather than using numbers throughout the code. That way, you give meaning to that number; if the number changes, you have to change it only at a single location.\n",
"- Avoid the use of globals. If a function needs a parameter, include it in the function call. If you must use globals, write the variable names in capitals. The risk of using globals is that if their value changes, it affects functions that depend on them while that dependency is hidden.\n",
"- When writing your code, be sure to decouple each function, test it separately, and briefly report on these tests. If you run into any problems further down the design process, finding where functions might not agree and where your problem could lie will be easier.\n",
"\n",
"The test itself should also be in a script, so that you can frequently rerun it in case some of the functions have changed and need to be tested again. \n",
"- In your report, describe the overall structure of the code and the main variables so that others can quickly find their way into your code.\n",
"- Test every function in your code! For every 'if' statement in the code, you have two branches to test.\n",
"\n",
"### Useful modules\n",
"\n",
"**Time measurement in Python** In IP3, accurately measuring time is crucial for various tasks such as determining the duration of events or operations. Python provides the *time* package, which offeers functionality to measure time intervals.\n",
"\n",
"To measure time intervals using *time* package, follow these steps:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import time\n",
"\n",
"# Record the start time\n",
"start = time.time()\n",
"\n",
"# Perform an operation or task\n",
"# For example, simulate a computational task\n",
"for _ in range(1000000):\n",
" pass\n",
"\n",
"# Record the end time\n",
"end = time.time()\n",
"\n",
"# Calculate the duration of the operation\n",
"duration = end - start\n",
"\n",
"# Print the duration\n",
"print(f\"The operation lasted for {duration} seconds.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this code snippet, the *time* package was imported. The *time.time()* function returns your computers time in seconds. The *time.time()* is called to record the start time before executing the operation. After completing the operation, the end time is recorded. By subtracting the start time from the end time, the duration of the operation is calculated.\n",
"\n",
"**Detecting Keyboard events in Python** In module 1, you will be tasked with controlling the car from your keyboard. For this you will need to detect keyboard events, such as key presses. \n",
"\n",
"To detect keyboard events using the *pynpyt* library, follow these steps:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from pynput import keyboard\n",
"\n",
"# Define callback for when a key is pressed\n",
"def on_press(key):\n",
" try:\n",
" print(f'Key {key.char} pressed')\n",
" except AttributeError:\n",
" print(f'Special key {key} pressed')\n",
"\n",
"# Define callback for when a key is released\n",
"def on_release(key):\n",
" print(f'Key {key} released')\n",
" # Stop listener if 'Esc' key is released\n",
" if key == keyboard.Key.esc:\n",
" return False\n",
"\n",
"# Setup the listener\n",
"with keyboard.Listener(on_press=on_press, on_release=on_release) as listener:\n",
" listener.join()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Explanation:\n",
"1. **`on_press` function:** This function is called whenever a key is pressed. It tries to print the character of the key (for normal alphanumeric keys). If it's a special key (like 'Shift', 'Ctrl', etc.), it will print the special key.\n",
"2. **`on_release` function:** This function is called whenever a key is released. It prints which key was released and stops the listener when the 'Esc' key is released.\n",
"3. **`keyboard.Listener`:** This is the main object that listens for keyboard events. The `on_press` and `on_release` functions are passed as callbacks.\n",
"\n",
"Run the script, and as you press or release keys, it will print the corresponding information to the console. The program will keep running until you press the 'Esc' key."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## KITT Simulator\n",
"\n",
"To facilitate the development and testing of your algorithms, we have created a digital model of it. This is a software tool that replicates the behavior of KITT, providing a virtual environment where you can test and refine your code without needing access to the physical car. It simulates the car’s dynamics, including speed, steering, and sensor responses, reacting to the same Python commands that control the real car. This allows you to experiment with different control algorithms and get immediate feedback on how the car would behave, even when you're away from the lab. Switching between the simulator and the physical car is seamless—simply change a single import statement at the top of your code to toggle between them.\n",
"\n",
"The car simulator is in the Python package `KITT_Simulator`. Its use is explained in Module 1.\n",
"\n",
"Below is already a small demo of what it can do:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from KITT_Simulator.serial_simulator import Serial\n",
"import time\n",
"# Open serial port\n",
"serial = Serial('/dev/ttyUSB0', 115200)\n",
"\n",
"# Wait for one second to allow for computer processing delay\n",
"time.sleep(1)\n",
"# Set speed and direction\n",
"serial.write(b'M163\\n')\n",
"serial.write(b'D150\\n')\n",
"\n",
"time.sleep(2)\n",
"\n",
"# Set speed and direction\n",
"serial.write(b'M162\\n')\n",
"serial.write(b'D120\\n')\n",
"\n",
"time.sleep(2)\n",
"\n",
"# Set speed to zero\n",
"serial.write(b'M150\\n')\n",
"print(\"Motors are OFF\")\n",
"\n",
"time.sleep(5) # Notice the car keeps moving (rolling out till standstill)\n",
"\n",
"# Close the connection (important!)\n",
"serial.close()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Limitations\n",
"\n",
"The simulator is based on the behavior of one specific car. It is important to note that each car has slight variations, which can lead to noticeable differences in performance. Factors like battery status, friction and motor efficiency can affect the accuracy of distance estimations and other dynamics. The simulator is best used for testing basic functionality—ensuring your code compiles, runs correctly, and performs simple tasks. However, the final validation should always be conducted on the real car. Additionally, some features in the simulator are simplified or idealized, such as distance sensors and audio recordings, which are based on generic settings rather than your specific configuration.\n",
"\n",
"### How the Car Simulator Works\n",
"\n",
"The car simulator consists of several key components that work together to create a realistic and interactive testing environment:\n",
"\n",
"1. **Dynamics Simulation Module**: This module models the physical behavior of the car, responding to inputs like throttle and steering. It factors in elements such as acceleration, deceleration, turning radius, and friction, providing realistic feedback on how the car would respond in different situations.\n",
"2. **Serial and State Communication**: Commands can be sent to the simulator via a serial interface or a shared state file. This enables smooth communication between your Python code and the virtual car.\n",
"3. **Sensor Simulation**: The simulator mimics the data output from physical sensors, including distance measurements and other feedback. This allows you to develop and test algorithms for tasks like feedback control and obstacle avoidance in a simulated environment.\n",
"4. **Graphical User Interface (GUI)**: The GUI offers a visual representation of the car’s movements, making it easy to observe its behavior in real time. You can monitor important metrics such as speed, distance traveled, and steering angle directly from the interface.\n",
"5. **Python Integration**: The simulator is designed to be controlled using Python scripts, just like the actual car. This ensures that the code you develop for the simulator can be easily adapted for use with the physical car, minimizing the transition effort. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "OpenEdVenv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

745
3_Module_1.ipynb Normal file
View File

@ -0,0 +1,745 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](pictures/header.png)\n",
"\n",
"[Table of Contents](0_Table_of_Contents.ipynb)\n",
"\n",
"# Chapter 3: Module 1 - Connecting with and Controlling KITT\n",
"\n",
"**Contents**\n",
"\n",
"* [Brief Overview](#brief-overview)\n",
"* [Communicating with KITT](#communicating-with-kitt)\n",
"* [Implementing a Basic KITT Control Script](#implementing-a-basic-kitt-control-script)\n",
"* [FAQ](faq)\n",
"\n",
"In this module, we will start working with the car simulator and the car. We will provide an overview and a brief description of the vehicle, and implement some basic control scripts."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"# Import necessary libraries\n",
"import time\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# Uncomment one of the following lines depending on your setup\n",
"\n",
"# If you are using the real car, uncomment the next line and comment the simulator line\n",
"# from serial import Serial\n",
"\n",
"# If you are using the simulator, uncomment the next line and comment the real car line\n",
"from KITT_Simulator.serial_simulator import Serial\n",
"\n",
"# Note: After changing the import statement, you need to restart the kernel for changes to take effect."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Brief Overview\n",
"\n",
"KITT is a vehicle that you can control remotely, similar to a typical RC car. While traditional RC cars usually come with a joystick for control, this project has adapted the car so that you can use your computer instead. You can program your computer and use your keyboard to make KITT move forward, backward, or turn.\n",
"\n",
"However, the goal of the project is to automate KITT, making it drive autonomously. As your computer will do the calculations, you will need to connect the car wirelessly to your computer. Over the wireless link, you can send commands to KITT, such as controlling its direction or speed. You can also request sensor data, which is a key difference from conventional RC cars. So this setup is a **two-way** communication link.\n",
"\n",
"In this module, you will learn how to set up this wireless connection and start controlling KITT using basic keyboard commands. We will use object-oriented programming. We will also do this in such a way that, by changing one line, you can swap the real car for a simulated (model) car, allowing you to test your scripts in a simulation before going into the field.\n",
"\n",
"### KITT Basics\n",
"\n",
"KITT is built on a modified toy car model, the Traxxas E-MAXX (see picture below). In addition to its original components, KITT is equipped with two ultrasonic sensors, an LCD display, and the necessary electronics to transmit data via a Bluetooth connection. The LCD on the car provides real-time information, such as the readings from the distance measurement sensors and the battery voltage level.\n",
"\n",
"In the Files>Datasheets>KITT datasheet.pdf, you can find a data sheet that gives a more detailed overview of the various components, such as the motor controller, LCD status indicator, ultrasonic sensors, and, most importantly, the communication module.\n",
"\n",
"**Note:** As a heads-up for the future, when you start gathering data from KITT via the wireless link, you might notice that it doesn't always match what’s shown on the car's LCD. The data received directly from the car is more accurate, as the LCD might update at a slower rate. Therefore, while the LCD can serve as an initial indicator of any issues, it's best to rely on the direct data from the car for precise information.\n",
"\n",
"<img src=\"pictures/traxxas_e-maxx.png\" alt=\"Traxass E-MAXX along with it's dimensions\" width=\"600\" />\n",
"\n",
"#### Hardware\n",
"\n",
"KITT is equipped with a custom-made circuit board that brings together various components needed for its operation. At the heart of this board is a microcontroller, which acts as the main control unit.\n",
"\n",
"This board also includes everything necessary for wireless communication, like the Bluetooth module, as well as connectors for other parts of the car. Additionally, there's an amplifier on the board that's used for sending out audio signals that help with localization.\n",
"\n",
"The circuit board is powered by rechargeable batteries. These batteries supply the necessary voltage through a component that adjusts the voltage to the required levels.\n",
"\n",
"Here are the key components included on the board:\n",
"\n",
"- **Microcontroller:** NXP LPC4357 chip with an ARM CORTEX-M4/M0 core\n",
"- **Bluetooth module**\n",
"- **Amplifier for the audio beacon**\n",
"- **Buck converter** for voltage regulation\n",
"- **Voltage regulator:** LM1117-3.3 for generating 3.3V DC\n",
"\n",
"<img src=\"pictures/communication_overview.png\" alt=\"Schematic of communication with KITT and PC.\" width=\"600\" />\n",
"\n",
"KITT has a Bluetooth module for communication with your PC. It consists of the following elements:\n",
"\n",
"- On car: Roving Networks RN-41 I/RM, onboard Bluetooth module with UART control.\n",
"- On PC: LM Technologies LM506, USB Bluetooth V4.0 dongle with Broadcom BCM20702 chipset, or the internal Bluetooth module of your laptop.\n",
"\n",
"**Tip:** If you want to use one of the lab computers or have trouble connecting to KITT from your PC (check if all drivers are installed), you could use the USB Bluetooth dongle.\n",
"\n",
"KITT has four LEDs on the front, see the figure below. Their meaning is as follows:\n",
"\n",
"<img src=\"pictures/figurejoin.png\" alt=\"KITT control board and front LEDs\" width=\"600\" />\n",
"\n",
"- **Red (twice):** *5V* and *20V* supply voltages are present.\n",
"- **Green (blinking):** Bluetooth is searching for a connection; **(steady):** Bluetooth connected.\n",
"- **Yellow (blinking):** Bluetooth data transfer.\n",
"\n",
"The button next to the LEDs is a reset button for the MCU. This reset button allows the user to reset KITT's actions if it freezes or is unresponsive.\n",
"\n",
"#### Software Model\n",
"\n",
"The KITT test fields have limited availability. To facilitate software development and testing, you can use a KITT software model. It is in the package `KITT_Simulator`. If you use the `Serial` command from this package and connect to KITT (see below), then instead of connecting to the real car, you connect to the software model. You can send commands and read its status using the same commands as for the real KITT. Thus, you can first test all your algorithms on the simulated model and later on the real KITT.\n",
"\n",
"Of course, this only works if the model is sufficiently close. At some point, you'll have to adjust the parameters in the code to match the reality of your car (calibration).\n",
"\n",
"To toggle between running the simulated model or the real car, look at the top block of this notebook: by commenting or uncommenting the appropriate import line, you can switch between the real car and the simulator. You will need to restart the kernel after changing this command.\n",
"\n",
"\n",
"\n",
"## Communicating with KITT\n",
"\n",
"In this section, we cover the essential concepts required to effectively communicate with the KITT system. In the following section, we will apply these principles in practice, implementing them in the simulator and the car.\n",
"\n",
"#### pySerial\n",
"\n",
"`pySerial` is a Python module that provides a simple and efficient way to communicate with serial ports. It allows Python programs to access and manipulate the serial ports on a computer, allowing them to communicate with other devices connected to those ports. With `pySerial`, you can easily send and receive data to/from these devices. `pySerial` is cross-platform and works on Windows, macOS, and Linux operating systems.\n",
"\n",
"- `Serial(port, baudrate)` - This command initializes a serial connection. The `port` argument specifies the serial port to use (e.g., \"COM1\" on Windows or \"/dev/rfcomm0\" on Linux), and the `baudrate` argument specifies the data rate in bits per second.\n",
"- `serial.write(data)` - This command sends data over the serial connection. The `data` argument is a bytes object that contains the data to be sent.\n",
"- `serial.read(size)` - This command reads a specified number of bytes from the serial connection. It blocks until the specified number of bytes are received.\n",
"- `serial.read_until(bytes)` - This command reads bytes from the serial connection until the specified byte sequence is found. It blocks until then.\n",
"- `serial.flush()` - This command is used to flush the input and output buffers of the serial connection.\n",
"- `serial.close()` - This command is used to close the serial connection.\n",
"\n",
"More detailed and updated information regarding the pySerial module can also be found through this link: [pySerial Documentation](https://pyserial.readthedocs.io/en/latest/pyserial.html). See the FAQ at the end of this file for some further explanations on serial communication. Regarding the use of the Serial(port,baudrate) command for our purposes, more specific information appears at the end of this file under \"Connecting to KITT\".\n",
"\n",
"#### Simulated Serial Port\n",
"\n",
"The KITT simulator (car model) is also controlled by a serial port. This port uses the following functions:\n",
"\n",
"- `Serial(port, baudrate)`\n",
"- `serial.write(data)`\n",
"\n",
"These functions work identically to the `pySerial` functions, except that they communicate with the simulated KITT and not the real KITT.\n",
"\n",
"#### Controlling KITT\n",
"\n",
"After connecting to KITT, several commands are available to control it: driving commands, audio beacon commands, and status commands.\n",
"\n",
"**Driving Commands:** For driving, there are two types of instructions:\n",
"\n",
"- A direction command: `D`. Example: `D130`\n",
"- A motor speed command: `M`. Example: `M160`\n",
"\n",
"These control the motors using Pulse Width Modulation (PWM). Both commands are neutral at a setting of 150. The direction commands range from 200 (hard left) to 100 (hard right), and the motor commands range from 135 (backward) to 165 (forward).\n",
"\n",
"**Note:** There is a dead zone for the motor commands, so KITT will likely not start moving forward until the PWM is set to about 153. It is recommended to experiment with these values; they are also battery-dependent. You should also verify that 150 is the middle position for steering (sometimes there is a deviation of ±2), and test the maximum left and right positions.\n",
"\n",
"All the commands are sent in binary over the serial link and end with a new line character.\n",
"\n",
"```python\n",
"serial.write(b'code\\n')\n",
"```\n",
"\n",
"To set direction to 130 and motor speed to 160, it can be done as follows:\n",
"\n",
"```python\n",
"serial.write(b'D130\\n')\n",
"serial.write(b'M160\\n')\n",
"```\n",
"\n",
"**Note:** Once you set the motor speed commands, KITT will continue to act on this until you either transmit a new motor speed command or reset the MCU using the button on KITT, which will set both direction and motor speed commands to the neutral value of 150. To be safe, include suitable stop commands in your scripts.\n",
"\n",
"**Simulator Example:** Let's see this in the simulator:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version: play with this! ###\n",
"\n",
"# Open the serial port with the correct port and baud rate\n",
"# Replace '/dev/ttyUSB0' with your actual serial port\n",
"serial = Serial('/path/to/serial/port', 115200)\n",
"\n",
"# Allow time for any processing delays\n",
"time.sleep(1)\n",
"\n",
"# TODO: Set a motor speed and direction\n",
"# Use serial.write() to send the motor speed command\n",
"# Example: serial.write(b'M<speed_value>\\n')\n",
"# Use serial.write() to send the direction command to turn left\n",
"# Example: serial.write(b'D<direction_value>\\n')\n",
"# Hint: To turn left, set the direction value greater than 150 (neutral position)\n",
"print(\"Motors are ON\")\n",
"\n",
"# Wait for some time to observe the car's movement\n",
"time.sleep(2)\n",
"\n",
"# Stop the car by setting motor speed to neutral (150)\n",
"# Use serial.write() to send the stop command\n",
"serial.write(b'M150\\n')\n",
"print(\"Motors are OFF\")\n",
"\n",
"# Wait to observe the car slowing down\n",
"time.sleep(5) # Notice the car keeps moving due to inertia\n",
"\n",
"# Close the serial connection (important!)\n",
"serial.close()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Additional Information:**\n",
"\n",
"*To make the car turn left instead of right, adjust the direction value in the `D` command accordingly. Remember that direction values range from 100 (hard right) to 200 (hard left), with 150 being neutral.*\n",
"\n",
"Also, notice that the car keeps moving after the motors are off. This is because of the car's dynamics, which will be explored in a later module.\n",
"\n",
"**Audio Beacon**\n",
"\n",
"The car has an audio beacon which will be used to localize the car. \n",
"The audio beacon, once switched on, continuously transmits a sequence of pulses. It is controlled by a programmable microcontroller on the car, and several parameters can be set. \n",
"\n",
"The pulse sequence is a modulated binary code sequence with “on-off keying” (OOK). If a bit in the\n",
"sequence is 0, nothing is transmitted; if the bit is 1, a modulation carrier frequency is transmitted during\n",
"a certain period. The bit sequence (code word) is programmable.\n",
"\n",
"Other parameters that determine the signal are\n",
"\n",
"- Modulation carrier frequency (parameter Timer0), at most 30 kHz, although this is probably beyond the specs of the loudspeaker and microphones;\n",
"- Duration of a single bit (parameter Timer1), this defines the rate at which the modulation carrier signal is switched on or off by the bits in the code word, and determines the bandwidth of the signal (at most 5 kHz);\n",
"- Repetition rate of the bit sequence (parameter Timer3). After the code sequence has been played over the loudspeaker, it will be silent for a certain period, and then the sequence is transmitted again, at a rate determined by Timer 3.\n",
"\n",
"The beacon parameters can be programmed using the following commands:\n",
"\n",
"- `F`: Set transmission frequency in Hz (controls Timer 0) - carrier frequency\n",
"- `B`: Set bit frequency in Hz (controls Timer 1) - determines the signal bandwidth\n",
"- `R`: Set repetition count (controls Timer 3) - determines the number of pulses per second\n",
"- `C`: Set 32-bit code - 8 characters representing Hex values\n",
"- `A`: Turn beacon on/off\n",
"\n",
"You can set the beacon parameters using the following commands:\n",
"\n",
"```python\n",
"serial.write(b'F10000\\n')\n",
"serial.write(b'B5000\\n')\n",
"serial.write(b'R2500\\n') \n",
"serial.write(b'C12345678\\n')\n",
"serial.write(b'A1\\n') # Turn beacon on\n",
"serial.write(b'A0\\n') # Turn beacon off\n",
"```\n",
"\n",
"<img src=\"pictures/Beacon.png\" alt=\"Illustration of the transmitted signal\" width=\"400\" />\n",
"\n",
"The repetition count formula is as follows:\n",
"\n",
"$$\\text{Repetition Count} = \\frac{\\text{Bit Frequency}}{\\text{Repetition Frequency}}$$\n",
"\n",
"For example, if the bit frequency is 5000 Hz, the repetition count should be 2500 to sound the beacon twice per second.\n",
"\n",
"Later, you will use this beacon to determine where your car is on the field. Remember that all the numbers provided here serve only as examples. It is up to you to determine what carrier frequency, code word, etc., best fits the goal of succeeding at the final challenge.\n",
"\n",
"Let's use these in the simulator. Unfortunately, the sound of the car cannot be heard in the simulation. A blue circle will appear on the car to indicate the when the beacon is on."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# Open the serial port with the correct port and baud rate\n",
"# Replace '/dev/ttyUSB0' with your actual serial port\n",
"serial = Serial('/path/to/serial/port', 115200)\n",
"\n",
"# Set carrier frequency\n",
"# Define the desired carrier frequency in Hz (e.g., 10000)\n",
"carrier_frequency_value = 10000 # Replace with your value\n",
"# Convert the frequency to bytes (2 bytes, big endian)\n",
"carrier_frequency = carrier_frequency_value.to_bytes(2, byteorder='big')\n",
"# TODO: Send the 'F' command with the carrier frequency\n",
"\n",
"# Set bit frequency\n",
"# Define the desired bit frequency in Hz (e.g., 5000)\n",
"bit_frequency_value = 5000 # Replace with your value\n",
"# Convert the bit frequency to bytes\n",
"bit_frequency = bit_frequency_value.to_bytes(2, byteorder='big')\n",
"# TODO: Send the 'B' command with the bit frequency\n",
"\n",
"# Set repetition count\n",
"# Calculate the repetition count (should be at least 32)\n",
"# Repetition Count = Bit Frequency / Repetition Frequency\n",
"desired_repetition_frequency = 2 # Replace with your value\n",
"repetition_count_value = bit_frequency_value // desired_repetition_frequency\n",
"# Ensure repetition_count_value is at least 32\n",
"repetition_count_value = max(repetition_count_value, 32)\n",
"# Convert the repetition count to bytes\n",
"repetition_count = repetition_count_value.to_bytes(2, byteorder='big')\n",
"# TODO: Send the 'R' command with the repetition count\n",
"\n",
"# Set code pattern\n",
"# Define a 32-bit code in hexadecimal (e.g., 0x12345678)\n",
"code_value = 0x12345678 # Replace with your code\n",
"# Convert the code to bytes (4 bytes, big endian)\n",
"code = code_value.to_bytes(4, byteorder='big')\n",
"# TODO: Send the 'C' command with the code\n",
"\n",
"# TODO: Allow some time for the settings to take effect\n",
"\n",
"# TODO: Turn the beacon ON by sending the 'A1' command\n",
"print(\"Beacon is ON\")\n",
"\n",
"# Wait while the beacon is on\n",
"time.sleep(3)\n",
"\n",
"# TODO: Turn the beacon OFF by sending the 'A0' command\n",
"print(\"Beacon is OFF\")\n",
"\n",
"# Allow some time before closing\n",
"time.sleep(1)\n",
"\n",
"# TODO: Very Important! Close the serial connection"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Additional Information:**\n",
"\n",
"*Remember to choose appropriate values for the carrier frequency, bit frequency, and code pattern. The carrier frequency should not exceed 20 kHz, and the bit frequency should be smaller than the carrier frequency. The code pattern is a 32-bit value that will be transmitted bit-wise over the beacon.*\n",
"\n",
"**Caution:** Be aware that the default code word for the beacon is `0x00000000`, which means KITT will not start making noise on its own when the beacon is turned on. You should specify a code as described above before you can hear the beacon make noise. Furthermore, an arbitrary carrier frequency, bit frequency, and repetition count can be used, but not all combinations make sense. (The carrier frequency should not be above 20 kHz; the bit frequency should be smaller than the carrier frequency.)\n",
"\n",
"**Status Command**\n",
"\n",
"Lastly, the status command, which requests the status and all the data from KITT, is `S`. The status string reports the current drive commands, the ultrasonic sensor distance (cm), the battery voltage (mV), the audio beacon status (on/off), and control parameters (code word, carrier frequency, bit frequency, repetition count). A sensor distance of 999 means overload (i.e., out of range).\n",
"\n",
"**Additional Information:**\n",
"\n",
"You can request information from KITT by sending one of the following commands:\n",
"\n",
"- **For all information**: Use the command:\n",
" ```python\n",
" serial.write(b'S\\n')\n",
" ```\n",
"\n",
"- **For distance measurements only**: Use the command:\n",
" ```python\n",
" serial.write(b'Sd\\n')\n",
" ```\n",
"\n",
"- **For battery voltage**: Use the command:\n",
" ```python\n",
" serial.write(b'Sv\\n')\n",
" ```\n",
"\n",
"- **For version information**: Use the command:\n",
" ```python\n",
" serial.write(b'V\\n')\n",
" ```\n",
"\n",
"After sending each command, you have to read out what KITT sends back. The response is a string of bytes that you should best decode before interpreting it.:\n",
"\n",
"```python\n",
"status = serial.read_until(b'\\n')\n",
"status = status.decode('utf-8')\n",
"print(status)\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Open the serial port with the correct port and baud rate\n",
"\n",
"# TODO: Send the status command 'S\\n'\n",
"\n",
"# TODO: Read the response until the End-of-Transmission character (EOT): '\\x04'\n",
"\n",
"# TODO: Decode the status information received from the car\n",
"\n",
"# Print the status information received from the car\n",
"print(f\"Car status is:\\n\\n{status}\")\n",
"\n",
"# TODO: Close the serial connection (important!)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Implementing a Basic KITT Control Script\n",
"\n",
"Now you can start writing a Python program to control the simulator or the car connected over Bluetooth. This program will allow you to control the car using the keyboard keys `w`, `a`, `s`, and `d`. The `e` and `r` keys will start and stop the audio beacon, respectively. Additionally, include a key to stop communication with KITT.\n",
"\n",
"The code block below gives a template for you to start with. We begin by defining the `KITT` class and some essential methods for communication. Classes define the structure and capabilities of objects, allowing you to create multiple instances with the same properties and methods. Next, you will complete the `keyboard_control` function. We'll use a simple input loop to read keyboard commands. You will finally call the function and test your code first on the simulator and then on the real car.\n",
"\n",
"### Explanation of the Code\n",
"\n",
"**KITT Class:**\n",
"\n",
"- **`__init__` Method:**\n",
" - Initializes the serial communication with the specified port and baud rate.\n",
" - Sets up the beacon parameters using the existing `send_command` method.\n",
"- **`send_command` Method:**\n",
" - Sends a given command to KITT over the established serial connection.\n",
"- **`set_speed` and `set_angle` Methods:**\n",
" - Adjust KITT’s speed and steering angle, respectively.\n",
"- **`stop` Method:**\n",
" - Brings KITT to a halt by setting both speed and angle to their neutral states.\n",
"- **`start_beacon` and `stop_beacon` Methods:**\n",
" - Control the audio beacon by sending the appropriate commands.\n",
"- **`close` Method:**\n",
" - Ensures the proper closure of the serial communication when you're done using KITT.\n",
"\n",
"**`keyboard_control` Function:**\n",
"\n",
"- Designed as a continuous loop that reads keyboard commands from the user input.\n",
"- Interprets the input command and adjusts KITT’s speed and steering angle or toggles the beacon accordingly.\n",
"- **Key Mappings:**\n",
" - `w`: Accelerate KITT forward.\n",
" - `s`: Decelerate or move KITT backward.\n",
" - `d`: Turn KITT right.\n",
" - `a`: Turn KITT left.\n",
" - `e`: Start the audio beacon.\n",
" - `r`: Stop the audio beacon.\n",
" - `q`: Stop communication with KITT and exit the program.\n",
"- Ensures that speed and steering values stay within the valid ranges.\n",
"- Handles exceptions and ensures resources are properly closed when exiting.\n",
"\n",
"**Test Code:**\n",
"\n",
"- The `if __name__ == \"__main__\"` block seen in the example code below contains the code that will execute when you run your script. This `if` construct allows the script to contain test code that will run only if the script is called directly.\n",
"- An instance of `KITT` is created with the appropriate serial port.\n",
"- The `keyboard_control` function is called with the `kitt` instance to start controlling the car.\n",
"\n",
"**Steps:**\n",
"\n",
"1. **Modify the `__init__` Method:**\n",
" - When the communication with KITT is started, initialize the beacon with the correct set of parameters using the existing `serial` port and `send_command`.\n",
"2. **Add `start_beacon` and `stop_beacon` Methods:**\n",
" - These methods will turn the beacon on or off. Since you set the beacon parameters during `__init__`, there is no need to resend them every time you turn the beacon on.\n",
"3. **Implemented a Simple Input Loop:**\n",
" - Using `input()` to read commands avoids issues with keybindings in VSCode or Jupyter notebooks.\n",
" - This method is more compatible and less likely to cause kernel crashes.\n",
"4. **Handled Exceptions and Resource Cleanup:**\n",
" - Added a `try...except...finally` block to handle any exceptions that might occur.\n",
" - Ensured that the car is stopped and the serial connection is closed when exiting the program."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"class KITT:\n",
" def __init__(self, port, baudrate=115200):\n",
" # Initialize the serial connection\n",
" # self.serial = Serial(port, baudrate, rtscts=True)\n",
" \n",
" # Initialize beacon parameters here using send_command\n",
" # Set carrier frequency, bit frequency, repetition count, and code pattern\n",
" pass # Replace with actual initialization code\n",
"\n",
" def send_command(self, command):\n",
" # Send the command string over the serial connection\n",
" self.serial.write(command.encode())\n",
" pass\n",
"\n",
" def set_speed(self, speed):\n",
" # Send the motor speed command using send_command\n",
" # Use the format 'M<speed>\\n'\n",
" pass\n",
"\n",
" def set_angle(self, angle):\n",
" # Send the steering angle command using send_command\n",
" # Use the format 'D<angle>\\n'\n",
" pass\n",
"\n",
" def stop(self):\n",
" # Stop the car by setting speed and angle to neutral (150)\n",
" # Call set_speed and set_angle with 150\n",
" pass\n",
"\n",
" def start_beacon(self):\n",
" # Send commands to start the beacon\n",
" # Use the command 'A1\\n'\n",
" pass\n",
"\n",
" def stop_beacon(self):\n",
" # Send commands to stop the beacon\n",
" # Use the command 'A0\\n'\n",
" pass\n",
"\n",
" def close(self):\n",
" # Close the serial connection\n",
" # self.serial.close()\n",
" pass"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"def wasd_control(kitt):\n",
" print(\"Control the car with W (forward), S (backward), A (left), D (right), E (start beacon), R (stop beacon), and Q (to exit).\")\n",
"\n",
" car_speed = 150 # Neutral speed\n",
" car_steering = 150 # Neutral steering\n",
"\n",
" try:\n",
" while True:\n",
" # Read input from the user\n",
" command = input(\"Enter command: \").lower()\n",
"\n",
" # TODO: depending on the command, adjust the car speed, steering angle, or start/stop the beacon variable\n",
"\n",
" # Ensure speed and steering values are within valid ranges\n",
" car_speed = min(max(car_speed, 135), 165)\n",
" car_steering = min(max(car_steering, 100), 200)\n",
"\n",
" # Send the speed and angle to KITT\n",
" kitt.set_speed(car_speed)\n",
" kitt.set_angle(car_steering)\n",
"\n",
" except Exception as e:\n",
" # Handle any exceptions\n",
" print(f\"An error occurred: {e}\")\n",
" finally:\n",
" # Ensure the car is stopped and serial connection is closed\n",
" kitt.stop()\n",
" kitt.close()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"if __name__ == \"__main__\":\n",
" # Create an instance of KITT with the correct serial port\n",
" # Replace '/dev/ttyUSB0' with your actual serial port\n",
" kitt = KITT('/path/to/serial/port') # Replace with the actual port\n",
"\n",
" # Start the control function\n",
" wasd_control(kitt)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Test the Code with the Real Car\n",
"\n",
"Now you can check and see if you can also control the real car with the code you wrote. For that, you need to restart the kernel and uncomment the appropriate `Serial` import at the top of the notebook. No code changes are required....\n",
"\n",
"However, theory and practice may differ. If you encounter any difficulties with connecting to KITT, read on.\n",
"\n",
"#### Connecting to KITT\n",
"To start with, connect the car to your computer over Bluetooth. Make sure you know the name of the Bluetooth port that the car is connected to. To access the link in Python, use:\n",
"\n",
"```python\n",
"serial_port = Serial(comport, 115200, rtscts=True)\n",
"```\n",
"\n",
"`serial_port` is an instance of `pySerial`, and this object can be used to manipulate KITT. The variable `comport` is the path to the port assigned to the Bluetooth connection with KITT. Which port specifically KITT is connected to can be found in your communication settings in Windows, etc.\n",
"\n",
"For example, if your transmission connection to KITT takes place over port 10 (Windows), then you should use:\n",
"```python\n",
"comport = 'COM10'\n",
"```\n",
"\n",
"Connecting to a serial port over Bluetooth can be tricky. Windows users, refer to Appendix B. Linux users can use `rfcomm` commands to bind the MAC address to a serial port; there is also a section about it on the [Arch Linux wiki](https://wiki.archlinux.org/title/Bluetooth#Bluetooth_serial). The MAC address of KITT always ends with the code printed on its label. For example, if the label reads DA:84, the MAC will end with `xx:xx:xx:xx:DA:84`.\n",
"\n",
"To find out which serial ports are available, you can use the following Python code:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import serial.tools.list_ports\n",
"\n",
"ports = list(serial.tools.list_ports.comports())\n",
"for port in ports:\n",
" print(port.device)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Note:** Keep in mind that the Bluetooth connection can be disrupted by leaving ports open, quitting your work without closing the communication link, removing the Bluetooth dongle, or turning off the Bluetooth connection. Avoid disturbing the Bluetooth connection by ending the connection properly before doing any of the aforementioned things. If you disturb the Bluetooth link, you may need to reboot your computer to reset the operating system, which costs you valuable lab time.\n",
"\n",
"The connection must also be closed once you are done using the port:\n",
"```python\n",
"serial_port.close()\n",
"```\n",
"This will release the resources, enabling another process to use the link. \n",
"\n",
"#### Optional Extensions\n",
"\n",
"If you would like to go above and beyond with the task, here are a few tips that can help with extending the functionality of the program:\n",
"\n",
"- **Add Error Handling:** Currently, if there is an error with the Bluetooth connection or the serial communication, the program will simply crash. You could add some error handling to handle these cases more gracefully, such as printing an error message and exiting the program.\n",
"- **Add Speed Control:** Currently, the program only supports moving the car forward or backward and turning left or right. You could add support for controlling the speed of the car, such as by sending different commands to the car depending on how long the user holds down the forward or backward keys.\n",
"- **Add an Emergency Stop:** KITT doesn’t have a brake; the 150 speed setting lets KITT roll out to standstill, which might take a long distance. Add an emergency brake by letting KITT drive backwards for a short period of time. (You should first detect if the previous speed setting caused KITT to move forward. You need to define a state variable to memorize the speed setting.)\n",
"\n",
"#### Mid-term Assessment and Report\n",
"\n",
"After you finish this assignment, and ultimately in week 4, showcase the functionality of your script to your assigned TA. After you pass this assessment, you are ready to document your results in your midterm report.\n",
"\n",
"For this module, you should include a chapter that covers the approach, implementation, testing, and results of the basic controller.\n",
"\n",
"Remember to document your code, using comments to define input/output variables of functions and to explain the logic and any modifications made. Your completed script will be crucial for the upcoming challenges, contributing to the overall autonomous driving system.\n",
"\n",
"After you have completed this module and have tested all the components thoroughly, start on the second part of the communication script outlined in Module 2."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Aeu7KHKYeh_l",
"jp-MarkdownHeadingCollapsed": true
},
"source": [
"## FAQ"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"**What is a serial communication and why is it used?**\n",
"\n",
"Serial communication is a way of transmitting data one bit at a time over a single communication line. It's commonly used to connect devices like computers and microcontrollers because it simplifies wiring and is efficient for long-distance communication. In this method, data is sent in a sequence, or \"serially,\" from the sender to the receiver, typically using standards like USB, RS-232, or UART."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"\n",
"**What is the COM port?**\n",
"\n",
"In a Bluetooth connection, a COM port (short for \"communication port\") is a virtual port that allows your computer to communicate with Bluetooth devices as if they were connected via a traditional serial port (like those used for older wired connections). When you pair a Bluetooth device with your computer, the operating system assigns it a virtual COM port, which acts as an interface for data exchange between your computer and the Bluetooth device. This makes it easier to send and receive data over Bluetooth using standard serial communication methods.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**What is a buck converter and why is it used?**\n",
"\n",
"A buck converter is a type of DC-DC converter that steps down voltage from a higher level to a lower level while maintaining efficient power transfer. It works by rapidly switching a transistor on and off to control the energy transferred to an output capacitor, which smooths out the voltage to a lower, steady level.\n",
"\n",
"Buck converters are used because they are highly efficient, meaning they waste very little power as heat. This efficiency is particularly important in battery-powered devices, like KITT, where it's crucial to conserve energy and maintain a stable voltage for sensitive electronics. For instance, if the battery supplies 12V but the circuit requires 5V, a buck converter efficiently reduces the voltage to the needed level without generating much heat or wasting energy.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"**I have paired the car with my PC, however the LED is still blinking; is the car connected?**\n",
"\n",
"Pairing your Bluetooth module and assigning a COM port is different from running the Python script. If you only pair the device, you'll notice that the LED continues to blink after some time. However, when you run the Python script and the virtual COM port is properly allocated, the LED will stop blinking. Simply pairing the device isn't enough to stop the LED from blinking—you need to run the Python script as instructed to see the LED become stable which means there is an actual connection."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SMvdbS610IJG"
},
"source": [
"**What is pulse width modulation?**\n",
"\n",
"Pulse Width Modulation (PWM) is a technique used to control the amount of power delivered to an electronic device by varying the width of the pulses in a signal. It works by switching a signal on and off rapidly at a constant frequency, and by adjusting the ratio of the \"on\" time to the \"off\" time, you can control the average power delivered to a load.\n",
"\n",
"For example, if a signal is \"on\" for 50% of the time and \"off\" for the other 50%, the output will deliver 50% of the maximum power. This method is commonly used in applications like controlling the speed of motors, dimming LED lights, and generating analog signals from digital sources because it is efficient and provides precise control over power.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JERUY0Yu0zil"
},
"source": [
"**I've implemented the code, and everything functions correctly in the GUI and runs without errors, but the car isn't moving. What should I do?**\n",
"\n",
"Keep in mind that there's an additional switch that controls the power to the motors. If this switch is turned off, the motors won't receive any power from the battery, even if everything else is set up correctly; the car won't move. Make sure this switch is turned on."
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "booktestenv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.4"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {}
}
},
"nbformat": 4,
"nbformat_minor": 4
}

729
4_Module_2.ipynb Normal file
View File

@ -0,0 +1,729 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](pictures/header.png)\n",
"\n",
"[Table of Contents](0_Table_of_Contents.ipynb)\n",
"\n",
"# Chapter 4: Module 2 - Reading KITT Sensor Data\n",
"\n",
"**Contents:**\n",
"* [Distance Sensor](#distance-sensors)\n",
"* [The Microphones](#the-microphones)\n",
"* [FAQ](#faq)\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import necessary libraries\n",
"import time\n",
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"import csv\n",
"\n",
"# Uncomment one of the following lines depending on your setup\n",
"\n",
"# If you are using the real car, uncomment the next lines and comment the simulator lines\n",
"# from serial import Serial\n",
"# import sounddevice\n",
"\n",
"# If you are using the simulator, uncomment the next lines and comment the real car lines\n",
"from KITT_Simulator.serial_simulator import Serial\n",
"from KITT_Simulator.sounddevice_simulator import sounddevice\n",
"\n",
"# Note: After changing the import statement, you need to restart the kernel for changes to take effect."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"KITT relies on its sensors to drive autonomously. It is equipped with:\n",
"1. Two front-mounted distance sensors.\n",
"2. Five microphones positioned around the field to record audio signals from KITT's beacon and relay them to the soundcard, after which they can be read by your PC.\n",
"\n",
"This task focuses on reading data from the distance sensors to avoid obstacles and processing the microphone data from the field.\n",
"\n",
"**Preparation**\n",
"- Ensure KITT is operational and properly set up.\n",
"- Reserve a time slot for testing on a field equipped with microphones and an audio card.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Distance Sensors\n",
"\n",
"KITT’s front distance sensors use ultrasonic technology. Two SRF02 modules, mounted on the left and right sides, measure the distance to obstacles. These \"parking sensors\" work by emitting a 40 kHz pulse and measuring the time it takes for the echo to return. This time is converted into a distance measurement.\n",
"\n",
"- Each sensor requires a minimum of 66 ms between readings, as specified in the SRF02 datasheet (available on Brightspace or at `Files/Datasheets/srf02.pdf`).\n",
"- The system is configured with a 70 ms cycle time; the left and right sensors take turns recording measurements.\n",
"- These measurements are stored in a buffer on KITT's microcontroller, with each new reading overwriting the previous one. When you request the KITT status, you will obtain a copy of the current buffer values.\n",
"\n",
"<img src=\"pictures/srf02-ultrasonic-sensor.jpg\" alt=\"Ultrasonic Sensor\" width=\"400\" height=\"240\">\n",
"\n",
"### Step 0: Characteristics of the Distance Sensors\n",
"\n",
"Using the readings on the car display, report on the following questions:\n",
"\n",
"1. What is the accuracy of the distance sensors? Does this change with distance?\n",
"2. What are the minimum and maximum distances the sensors can measure?\n",
"3. What is the field of view of the distance sensors (beam angle)?\n",
"\n",
"To measure this field of view, move an obstacle from left to right over a line, at about 1 m distance from the sensors, and observe when the sensors start to 'see' the object.\n",
"\n",
"The field of view is important when making recordings: you should realize that the distance sensors may detect chairs, bags, etc., and then make false readings. This happens even if these objects are not straight in front of the sensors. (The field of view does depend on distance.)\n",
"\n",
"**Note:** Do not copy the questions into your report, but naturally include the information in your report as part of your discussion.\n",
"\n",
"### Step 1: Status Command\n",
"\n",
"To ensure you can experiment at home, we have added the status command to the simulator. The simulator will accurately simulate the sensor distances, but not its behavior. Make sure to test on the real car frequently.\n",
"\n",
"As you have learned in the previous module, you can ask KITT to capture a status command by writing `\"S\\n\"` to the serial port. Then you have to read the message using `read_until`; this will generate a binary message that you need to decode. KITT always ends its message with the end-of-transmission character (0x04). The response contains three sections:\n",
"1. Audio beacon status and settings\n",
"2. PWM values for the motors\n",
"3. Sensor readings"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Establish a serial connection, ask for a status report, read it out, and print it\n",
"\n",
"# TODO: Close the serial connection"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you only need distance measurement information, you can request it separately:\n",
"\n",
"```python\n",
"serial.write(b'Sd\\n')\n",
"```\n",
"\n",
"This returns only the left and right distance sensor values, filtering out the rest of the status report.\n",
"\n",
"### Step 2: Extracting and Isolating Distance Data\n",
"\n",
"Assuming you have received the full status information from KITT, you can extract and isolate the distance sensor readings (left and right) from the status report.\n",
"\n",
"After sending the status command (`b'S\\n'`), the response will contain a variety of information, including the distance measurements. Now write a Python function to extract the distance data from the status report.\n",
"\n",
"1. **Extract the distance measurements**:\n",
"\n",
"The distance values are typically embedded in the `Sensors` section of the status response. You can process the `status` output to isolate just the left (`L`) and right (`R`) distance sensor values. Write a function to extract these values.\n",
"\n",
"*Hints:*\n",
"\n",
"- Use `decode('utf-8')` to convert bytes to a string.\n",
"- Use `splitlines()` to separate the status message into individual lines.\n",
"- Look for the line that contains `\"Dist.\"` to find the distance measurements.\n",
"- Use `split()` to break the line into individual words.\n",
"- Be cautious of the positions of the distance values in the list; adjust indices as necessary.\n",
"\n",
"**Note:** If you use the `Sd` status command, you retrieve less info and can write a faster function! The parsing of the status string is also easier."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Decode the status response to a string\n",
"\n",
"# TODO: Split the status string into lines\n",
"lines = \n",
"\n",
"# Initialize variables to hold distance values\n",
"dist_L = None\n",
"dist_R = None\n",
"\n",
"# Iterate over each line to find distance data\n",
"for line in lines:\n",
" if \"Dist.\" in line:\n",
" # TODO: Split the line into words\n",
" words =\n",
" # Extract distance values based on their positions\n",
"\n",
" # Assign dist_L and dist_R accordingly\n",
" dist_L = \n",
" dist_R = \n",
" break # Exit the loop after finding the distances\n",
"\n",
"# Print the extracted distance values\n",
"print(f\"Left Distance: {dist_L}\")\n",
"print(f\"Right Distance: {dist_R}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"2. **Determine how fast you can read out (and process) your distance data** by writing a script that requests the status 100 times. You can calculate the average delay for this operation (and its standard deviation); you could also present the results in a histogram. To measure delays, you will need to keep track of time:\n",
"\n",
"```python\n",
"start_time = time.time() # Initialize\n",
"current_time = time.time() - start_time # Find current time since initialization\n",
"```\n",
"\n",
"If you can read out the sensors faster than 70 ms (or is it 140 ms?), reason if you will obtain duplicate values from the buffer.\n",
"\n",
"**Student Task:**\n",
"\n",
"- Write a script that sends the status command 100 times, recording the time taken for each read.\n",
"- Store the time intervals in a list.\n",
"- After collecting the data, calculate the average delay and standard deviation.\n",
"- Plot a histogram of the delays."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 3: Using Distance Values to Model the Car\n",
"\n",
"Ultrasonic sensors are not just used for detecting obstacles; they play a crucial role in modeling the car's behavior during autonomous driving. To control the car effectively, we need to understand how it responds to drive and steering commands, similar to how a human driver knows how much acceleration or steering input affects the car's movement.\n",
"\n",
"However, while KITT doesn’t have an accelerometer to measure acceleration directly, we can use the ultrasonic sensors to estimate how the car moves over time. By measuring the distance to a cardboard-box wall, we can derive its speed and acceleration.\n",
"\n",
"#### Understanding Speed and Acceleration\n",
"\n",
"- **Velocity** is the rate of change of position over time:\n",
"\n",
" $$\n",
" v(t) = \\frac{{\\mathrm d} x}{{\\mathrm d} t}(t) \\,.\n",
" $$\n",
"\n",
"- **Acceleration** is the change in speed over time:\n",
"\n",
" $$\n",
" a(t) = \\frac{{\\mathrm d} v}{{\\mathrm d} t}(t) \\,.\n",
" $$\n",
"\n",
"Note that $x(t)$, $v(t)$ and $a(t)$ are time varying. To implement the differentials, in practice we will subtract two subsequent samples $x(t_1)$ and $x(t_2)$. We will then have an estimate of $v(t)$ for $t = (t_1+t_2)/2$:\n",
"\n",
"$$\n",
"v\\left(\\frac{t_1+t_2}{2}\\right) \\approx \\frac{x(t_2) - x(t_1)}{t_2-t_1} \\,.\n",
"$$\n",
"\n",
"In theory, this approximation gets better for $t_2$ close to $t_1$, but at the same time the division of two small numbers will make the result very sensitive to noise, so in practice there is a trade-off.\n",
"\n",
"#### Plotting KITT's Motion Towards a Wall\n",
"\n",
"To understand how KITT moves, make recordings of the distance sensor values as KITT drives towards a wall. Do this for multiple motor commands, and store them in a `.csv` file. (You can use the `Files/Recordings` folder to organize your data). You can then later import the data into Python. Next to the sensor values, you should also store the time stamp of each sample.\n",
"\n",
"You will see KITT speed up, and then reach a constant speed. To do this experiment, please let KITT drive towards the supplied cardboard wall. **Turn off KITT's motors once the distance is less than 40 cm to ensure KITT does not crash into the wall.** Note that you may have to discard the first few readings as they may be inaccurate.\n",
"\n",
"*Hints:*\n",
"\n",
"- Choose an appropriate motor speed value for `motor_speed_value`.\n",
"- Ensure that you stop the car if it gets too close to the wall to prevent collisions.\n",
"- Use `time.time()` to keep track of elapsed time.\n",
"- Store the data in a list with the format `[current_time, dist_L, dist_R]`.\n",
"- Write the data to a CSV file for later analysis.\n",
"- Also document the motor speed setting (e.g. use this as part of your file name)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Open the serial connection to KITT, set the motor speed\n",
"\n",
"# Initialize a list to store recorded data\n",
"data = []\n",
"\n",
"# Record data for a specified duration (e.g., 10 seconds)\n",
"recording_duration = 10 # in seconds\n",
"start_time = time.time()\n",
"\n",
"while time.time() - start_time < recording_duration:\n",
" # Send the status command to get the distance readings\n",
" serial_port.write(b'S\\n')\n",
" \n",
" # Read the status response\n",
" status = serial_port.read_until(b'\\x04').decode('utf-8')\n",
" \n",
" # TODO: Extract the distance values from the status response\n",
"\n",
" dist_L = None\n",
" dist_R = None\n",
" \n",
" # TODO: Record current time and distances\n",
" data.append([current_time, dist_L, dist_R])\n",
" \n",
" # Check if KITT is too close to the wall and stop if necessary\n",
" if dist_L < 40 or dist_R < 40:\n",
" serial_port.write(b'M150\\n') # Stop the car\n",
" print(\"Stopping KITT to avoid collision.\")\n",
" break # Exit the loop\n",
" # Note: you can also add a small loop here and still read the stopping data\n",
" \n",
" time.sleep(0.1) # Wait before the next reading\n",
"\n",
"# Close the serial connection\n",
"serial_port.close()\n",
"\n",
"# TODO: Write the recorded data to a CSV file\n",
"# Recommeded file output: Files/Recordings/kitt_distance_data_{speed}.csv"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Processing the Recorded Data\n",
"\n",
"Read your `.csv` data into Python (see template script below), and plot the distance values over time to visualize KITT's motion. Use a single plot with separate colors for the L and R sensors. You should notice a 'staircase' shape! Explain this in your report.\n",
"\n",
"Next, to derive velocity, first merge the L and R sensor data into a single position estimate: remove the duplicate values (keep only the first value of a duplicate reading), then merge the remaining values into a single (time, position) array. Plot the result in your distance plot to see if you did this correctly.\n",
"\n",
"After that, estimate the velocity of KITT as function of time. Obviously, you will use $ v(t) = \\Delta x / \\Delta t \\ $, but what time $t$ do you associate with each of these estimates?\n",
"\n",
"Make a plot of the resulting velocity estimates over time.\n",
"\n",
"*Hints:*\n",
"\n",
"- When merging the L and R distance measurements, don't simply average them. Read the above paragraph again.\n",
"- Be aware that the sensors alternate readings every 70 ms, leading to a 'staircase' effect.\n",
"- To calculate velocity, use the differences in distance and time (`diff()` function).\n",
"- Since KITT is moving towards the wall, the distance decreases; yet, the estimated velocity should be positive when moving forward.\n",
"- For the time associated with each velocity estimate, use the midpoint between consecutive time stamps.\n",
"- Remove any NaN values resulting from the `diff()` operation."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"motor_speed_value = 160 # Use the same motor speed as during recording\n",
"\n",
"# Load the recorded data from the CSV file\n",
"csv_filename = f'Files/Recordings/kitt_distance_data_{motor_speed_value}.csv'\n",
"data = pd.read_csv(csv_filename)\n",
"\n",
"# Consider discarding the first few readings (inaccurate readings)\n",
"\n",
"# TODO: Remove duplicate time stamps and merge L and R data\n",
"# Create a new DataFrame to hold your processed data\n",
"merged_data = []\n",
"\n",
"# Iterate over the data\n",
"for index, row in data.iterrows():\n",
" # Extract time and distances\n",
" time_stamp = row['Time']\n",
" dist_L = row['Distance_L']\n",
" dist_R = row['Distance_R']\n",
" \n",
" # TODO: Decide which distance to use or how to merge them\n",
" distance = \n",
"\n",
" merged_data.append([time_stamp, distance])\n",
"\n",
"# Convert merged data to DataFrame\n",
"merged_df = pd.DataFrame(merged_data, columns=['Time', 'Distance'])\n",
"\n",
"# TODO: calculate velocity (change in distance over change in time)\n",
"merged_df['Velocity'] = \n",
"\n",
"# Note: Use a negative sign because distance to the wall decreases as KITT moves forward\n",
"\n",
"# Calculate the time corresponding to each velocity estimate\n",
"# It's common to use the midpoint of the time intervals\n",
"merged_df['Velocity_Time'] = merged_df['Time'] - merged_df['Time'].diff() / 2\n",
"\n",
"# Plotting Distance\n",
"plt.figure()\n",
"plt.plot(merged_df['Time'], merged_df['Distance'], label='Distance to Wall')\n",
"plt.xlabel('Time (s)')\n",
"plt.ylabel('Distance (cm)')\n",
"plt.title('Distance to Wall Over Time')\n",
"plt.grid(True)\n",
"plt.legend()\n",
"plt.show()\n",
"\n",
"# Plotting Velocity\n",
"plt.figure()\n",
"plt.plot(merged_df['Velocity_Time'], merged_df['Velocity'], label='Velocity (cm/s)')\n",
"plt.xlabel('Time (s)')\n",
"plt.ylabel('Velocity (cm/s)')\n",
"plt.title('Velocity of KITT Over Time')\n",
"plt.grid(True)\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Tips to Consider:**\n",
"\n",
"**Continuous Measurement** involves data that can be taken at any point in time, with no gaps. For example, a car’s speedometer provides a continuous record of the car’s speed.\n",
"\n",
"**Discrete Measurement**, on the other hand, collects data at specific intervals. For instance, KITT’s ultrasonic sensors take distance readings every 70 ms. In between these measurements, we don’t know the exact position of the car. Discrete data can still be useful, but it may miss details about rapid changes in speed or acceleration that occur between measurements. In order to interpret it correctly, you may need to filter or interpolate the data.\n",
"\n",
"The following shows the difference between continuous and discrete data:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Define time for continuous measurement (smooth, no gaps)\n",
"time_continuous = np.linspace(0, 10, 1000) # Time from 0 to 10 seconds, 1000 data points\n",
"# Define time for discrete measurement (specific intervals)\n",
"time_discrete = np.linspace(0, 10, 20) # Time from 0 to 10 seconds, 20 data points\n",
"# Simulate continuous speed (sinusoidal speed pattern for illustration)\n",
"speed_continuous = 10 * np.sin(0.5 * np.pi * time_continuous) # Continuous speed\n",
"# Simulate discrete speed (sampled at specific intervals)\n",
"speed_discrete = 10 * np.sin(0.5 * np.pi * time_discrete) # Discrete speed\n",
"# Plotting both continuous and discrete measurements\n",
"plt.figure(figsize=(7, 4))\n",
"plt.plot(time_continuous, speed_continuous, label=\"Continuous Measurement\", color=\"blue\")\n",
"plt.scatter(time_discrete, speed_discrete, label=\"Discrete Measurement\", color=\"red\", zorder=5)\n",
"plt.xlabel('Time (s)')\n",
"plt.ylabel('Speed (cm/s)')\n",
"plt.title('Continuous vs Discrete Measurement of Speed')\n",
"plt.legend()\n",
"plt.grid(True)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 5: Implementing the Distance Sensor Reading in Your KITT Class\n",
"\n",
"In the previous module, you have created a class for KITT. Add a method to read the distance sensors to your `KITT` class in your 'Student Code' files. You can use the code you have written in the previous steps to do this. Make sure to test your code. It is advisable to store all the old distance data in a list inside the `KITT` class. This will be convenient during the final challenge, where the route planning might need old measurements to determine the position of objects.\n",
"\n",
"**Student Task:**\n",
"\n",
"- Add a method `read_distance_sensors()` to your `KITT` class.\n",
"- The method should send the status command to KITT and extract the distance measurements.\n",
"- Store the readings along with timestamps in an internal list or data structure.\n",
"\n",
"### Step 6: Mid-term Assessment 2.1 and Report\n",
"\n",
"After you finish this assignment, and ultimately in week 4, showcase the functionality of your script to your assigned TA. After you pass this assessment, you are ready to document your results in your midterm report. For this Module, you would include a chapter that covers the above tasks (using independently-readable text, i.e., don’t refer to “Step 1”).\n",
"\n",
"Include plots; for each plot, it should be clear how the plot was made (i.e., the corresponding experimental setup), and you have to describe what is seen in the plot before you discuss results and derive conclusions. Review the guidelines in Chapter 7 for more information. Include the corresponding code in an appendix.\n",
"\n",
"Remember to document your code, using comments to define input/output variables of functions and to explain the logic and any modifications made. Your completed script will be crucial for the upcoming challenges, contributing to the overall autonomous driving system."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The Microphones\n",
"\n",
"The field is equipped with four microphones at its corners, and a fifth microphone positioned at a higher level between two of the edge microphones. These microphones, together with the beacon mounted on KITT, will be used to locate KITT within the field (more details in Chapter 5).\n",
"\n",
"<img src=\"pictures/axisdef.png\" alt=\"Microphones Axis Defenition\" width=\"400\" height=\"240\">\n",
"\n",
"To use the microphone array, you must ensure that the correct sound card driver is installed. The sound card used in this project is the **Scarlett 18i20 3rd Gen**. Below are instructions on how to configure Sounddevice and the necessary drivers on different platforms.\n",
"\n",
"### Simulator\n",
"For the Sounddevice package another simulator has been made. The simulator will return a realistic audio recording, and change the recordings according to the location of the car. But, it contains only 1 recording, so it will not appear as random as the real car. It also does not adjust to your particular beacon settings. Make sure to test on the real car frequently. Use it in combination with the serial simulator to change locations and test like you would on the real car.\n",
"\n",
"### Important: Lab rules for the microphone array\n",
"\n",
"When working with the microphone array, please follow these rules to ensure smooth operations and avoid disrupting other groups:\n",
"\n",
"1. **Do not rearrange the microphone connectors**. The setup is shared between multiple groups, and changing the connections may lead to incorrect results for other teams.\n",
"2. **Do not touch the volume settings**. If the volume needs adjustment, contact a TA for assistance.\n",
"3. **Handle the equipment carefully**. The microphone array and associated hardware are sensitive, and mishandling could cause damage.\n",
"4. **Start and stop on time**. The lab is shared, and other groups have scheduled time slots. Be respectful of their time.\n",
"\n",
"Test time is limited. But by using the simulator and preparing a plan of what you want to test during each scheduled slot, there should be enough time to complete the tasks. \n",
"\n",
"### Step 1: Initializing the microphone array\n",
"\n",
"Before using the microphone array, it must first be initialized. As part of the initialization process, you will need to specify the sampling frequency (`Fs`) that will be used to record the audio. The sampling frequency will vary based on the test field you are working with, and it will be **48 kHz** or **44.1 kHz**.\n",
"\n",
"A typical laptop or PC may have multiple audio devices (e.g., built-in microphones, Bluetooth headsets, external sound cards). To ensure that the correct device is used, you can list all available audio devices using Sounddevice and select the appropriate one. Use the following code snippet to list all audio devices recognized by Sounddevice and find the index of the Scarlett 18i20 or any other relevant device:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"sounddevice_handle = sounddevice()\n",
"for i in range(sounddevice_handle.get_device_count()):\n",
" device_info = sounddevice_handle.get_device_info_by_index(i)\n",
" print(i, device_info['name'])\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once you have identified the index of the microphone array device from the list, you can initialize it by specifying the device index (`device_index`) and the desired sampling frequency (`Fs`).\n",
"\n",
"Here’s how you can open the audio stream using Sounddevice:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Initialize sounddevice again\n",
"sounddevice_handle = sounddevice()\n",
"\n",
"# Specify the device index and sampling frequency\n",
"device_index = 1 # Replace with the index of your microphone device\n",
"Fs = 44100 # or 48000, depending on the field setup. Try to use 48000 when making your own recordings.\n",
"\n",
"# Open the audio stream with 5 channels, 16-bit audio format (paInt16), and the specified sample rate\n",
"stream = sounddevice_handle.open(input_device_index=device_index,\n",
" channels=5,\n",
" format='float32',\n",
" rate=Fs,\n",
" input=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 2: Recording audio data\n",
"\n",
"To make a recording with the microphone array, you must specify the **length of the recording** as the number of **audio frames** to capture. Each audio frame consists of samples from all 5 microphones. Given that we are using 16-bit audio (2 bytes per sample), each frame will contain **10 bytes** (5 microphones × 2 bytes per sample).\n",
"\n",
"Thus, recording **N frames** will produce **10N bytes** of data. Note: The simulator returns a fixed length recording at 44.1 kHz. The real car will return a recording of the length you specify.\n",
"\n",
"The following command records `N` frames from the microphone array and stores the result as a numpy array:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"Fs = 44100 # Sampling frequency\n",
"N = 2*Fs # 2 seconds of audio data\n",
"samples,_ = stream.read(N)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"At this point, the microphone data is **interleaved**. This means that the first value (`data[0]`) corresponds to the first sample of microphone 0, the second value (`data[1]`) corresponds to the first sample of microphone 1, and so on. For example, `data[5]` contains the second sample of microphone 0, and the pattern continues. To visualize the interleaving of the data, refer to the table below:\n",
"\n",
"| data[0] | data[1] | data[2] | data[3] | data[4] | data[5] | data[6] | data[7] | ... |\n",
"|---------|---------|---------|---------|---------|---------|---------|---------|-----|\n",
"| mic 0 | mic 1 | mic 2 | mic 3 | mic 4 | mic 0 | mic 1 | mic 2 | ... |\n",
"| frame 0 | frame 0 | frame 0 | frame 0 | frame 0 | frame 1 | frame 1 | frame 1 | ... |\n",
"\n",
"#### Deinterleaving the data\n",
"\n",
"To work with the data from each microphone independently, the **interleaved data** must be split into separate streams for each microphone. This process is called **deinterleaving**.\n",
"\n",
"Write a function to deinterleave the audio data and store the samples from each microphone in a separate numpy array:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Reshape the data into a matrix with 5 columns (one for each microphone)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Plotting the audio data\n",
"\n",
"Once you've extracted the audio data for each microphone, you can plot it using Python. **Matplotlib** is a commonly used module for creating plots. Plot the audio data from each microphone to visualize the sound captured by the microphone array:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Plot the data for each microphone"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 3: Testing the microphone array\n",
"Below are some experiments you can perform to test the microphone array, and develop the code.\n",
"\n",
"**Part 1: Clapping**: With the real microphones connected, have your teammate clap in front of each microphone in turn. Record the audio data and plot the results. Is the order of the microphones what you expected? How does the sound intensity change as you move from one microphone to another?\n",
"\n",
"**Part 2: Beacon detection**: Turn on KITT's beacon and record the results. Can you identify where KITT is located just by observing the shift in the recordings? Change the beacons parameters and see how it affects the recordings.\n",
"\n",
"**Part 3: Ideal OOK signal**: Compare the waveform of the recording to an ideal OOK of your code. What can you see, and what do you infer from this? Are some beacon signals better than others? How can you find a good beacon signal? (This point is revisited in Module 3.)\n",
"\n",
"**Part 4: Reference recording**: Make some recordings of the beacon at different locations. These recordings will be useful to your teammates working on the localization algorithm. Similarly, make a recording of a single pulse from the beacon close to one of the microphones. Cut out the pulse and save it separately.\n",
"\n",
"**Part 5: KITT class**: Add a method to read the microphones to your KITT class in 'Student Code' files. The method should make a stream, turn on the beacon, start the recording, stop the recording, and turn off the beacon. You can choose to return the recording as a result, or store it internally inside the KITT class. Make sure to test your code."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Bonus Tasks - Optional*\n",
"\n",
"- See if you can automate selecting the correct sounddevice device index. The correct device index changes from one computer to another and can sometimes even change on the same computer after a reboot. So, it is worth your time to make a program that can automatically select the right device index.\n",
"- Implement start-up sanity checks: some process which you can run after you arrive at the test field, so that you can quickly check the microphone connections and audio levels.\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Mid-term assessment 2.2 and report\n",
"\n",
"After you finish this assignment, and ultimo in week 4, showcase the functionality of your script to your\n",
"assigned TA. After you pass this assessment, you are ready to document your results in your midterm\n",
"report.\n",
"\n",
"For this Module, you would include a chapter that covers the above tasks (using independently-readable\n",
"text, i.e., don’t refer to “Task 1”). Include plots; for each plot it should be clear how the plot was made\n",
"(i.e., the corresponding experimental set-up), and you have to describe what is seen in the plot before\n",
"you discuss results and derive any conclusions. Be sure to answer the questions posed along with the\n",
"plots (using independently-readable text).\n",
"\n",
"Include the corresponding code in an Appendix. Remember to document your code, using comments\n",
"to define input/output variables of functions and to explain the logic and any modifications made. Your\n",
"completed script will be crucial for the upcoming challenges, contributing to the overall autonomous\n",
"driving system.\n",
"\n",
"This concludes the mid-term assignments related to communication with KITT. After the mid-term, you\n",
"must integrate this module with the localization module created by your colleagues. Take into account\n",
"that integrating is often harder than originally anticipated, e.g. your code has to run in parallel, and you\n",
"have to worry about timing aspects. Hopefully, using the KITT class will provide you with a sturdy and\n",
"flexible framework to continue your work towards the final challenge\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## FAQ\n",
"\n",
"**What is the beam angle ?**\n",
"\n",
"The beam angle of a sensor refers to how wide the sensor's detection area is. It determines how much space the sensor can cover when it sends out signals (like sound or light) to detect objects.\n",
"\n",
"To determine the beam angle of ultrasonic sensors mounted in front of the car, you have multiple options: \n",
"\n",
"1. **Check the sensor datasheet**: The easiest way or at least a way to get some idea to determine the beam angle is to refer to the manufacturer's datasheet for your specific ultrasonic sensor. The datasheet will typically provide the beam angle, often around 15 to 30 degrees for common ultrasonic sensors. But keep in mind that is for a single sensor and not the current set up! Also, the 'reach' of the sensor is angle dependent: straight ahead, it can see several meters, but at an angle, perhaps just half a meter. \n",
"\n",
"2. **Experimental Determination for KITT**:\n",
" - **Measure detection width**: Place a flat object (like a wall) at a fixed distance in front of the sensor (e.g., 1 meter).\n",
" - **Move the object**: Move the object left and right to determine the points where the sensor stops detecting the object.\n",
" - **Calculate the angle**: Measure the distance between these two points (detection width) and the distance from the sensor to the object. You may use the following formula:\n",
"\n",
" \n",
" $$\n",
" \\text{Beam Angle} = 2 \\times \\arctan\\left(\\frac{\\text{Detection Width}/2}{\\text{Distance to Object}}\\right)$$\n",
" \n",
" - This calculation will give you the beam angle in degrees.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**I see random numbers from sensors for large distances. Is my sensor damaged ?**\n",
"\n",
"During experiments, you may occasionally receive random or unexpected data from the sensors. This can occur not only when the sensors are operating outside their effective range but also at times when they are within range. Several factors (consider what they might be?) can cause ultrasonic sensors to produce inaccurate readings. Additionally, since there are two sensors—one on the left and one on the right—they might produce different, completly different readings.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Are the ultrasonic sensor measurements for the left and the right side done at exactly the same time ?**\n",
"\n",
"If you closely observe the blinking of the small LEDs on the ultrasonic board on the car, you might notice that they turn on and off alternatingly. This indicates a slight time difference in the sensor measurements. This delay is also noticeable and can be measured using a moving car. (The reason to operate the two sensors alternatingly is that otherwise they might interfere on each other.)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

936
5_Module_3.ipynb Normal file
View File

@ -0,0 +1,936 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](pictures/header.png)\n",
"\n",
"[Table of Contents](0_Table_of_Contents.ipynb)\n",
"\n",
"# Chapter 5: Module 3 - Locating KITT Using Audio Communication\n",
"\n",
"**Contents:**\n",
"* [Pre-recorded Data](#pre-recorded-data)\n",
"* [Background Knowledge](#background-knowledge)\n",
"* [Step 1: Implementing Channel Estimation ](#step-1-implementing-channel-estimation)\n",
"* [Step 2: Determining Time of Arrivals](#step-2-determining-time-of-arrivals)\n",
"* [Step 3: Pulse Segmentation](#step-3-pulse-segmentation)\n",
"* [Step 4: Calculating TDOA Between Microphone Pairs](#step-4-calculating-tdoa-between-microphone-pairs)\n",
"* [Sanity Check](#sanity-check)\n",
"* [Localization Using TDOA Information](#localization-using-tdoa-information)\n",
"* [Localization Class](#localization-class)\n",
"* [Optional Extension](#optional-extensions)\n",
"* [Assessment and Reporting](#assessment-and-reporting)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import necessary libraries\n",
"import matplotlib.pyplot as plt # For plotting purposes\n",
"import numpy as np # For convolution function\n",
"from scipy.io import wavfile\n",
"from scipy.signal import find_peaks\n",
"import time\n",
"from scipy.fft import fft, ifft\n",
"\n",
"# Uncomment one of the following lines depending on your setup\n",
"\n",
"# If you are using the real car, uncomment the next lines and comment the simulator lines\n",
"# from serial import Serial\n",
"# from sounddevice import sounddevice\n",
"\n",
"# If you are using the simulator, uncomment the next lines and comment the real car lines\n",
"from KITT_Simulator.serial_simulator import Serial\n",
"from KITT_Simulator.sounddevice_simulator import sounddevice\n",
"\n",
"# Note: After changing the import statement, you need to restart the kernel for changes to take effect."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"KITT must be located in its field, and directions must be determined to navigate to the final destination. In previous modules, your colleagues have developed scripts to communicate with KITT and read sensor data. You will focus on locating KITT using audio signals received by microphones placed around the field. It is recommended that you read Modules 1 and 2 and talk to the other sub-group frequently. Otherwise the odds of your codes working together are small.\n",
"\n",
"For localization, we will use recordings of the beacon signal at various microphones, deconvolve these using a reference signal, and determine the relative time delays from the resulting channel estimates. These time differences, known as Time Difference of Arrival (TDOA), can be used to estimate KITT's position within the field. (You will be able to reuse your `ch2` or `ch3` code from the courselab Assignments.)\n",
"\n",
"At the end of this module, you will have developed a script to locate KITT within the field with reasonable accuracy, using the data recorded by the microphones. You will also have tested and verified the accuracy and robustness of your solution. \n",
"\n",
"The module is divided into the following 3 sections:\n",
"1. _TDOA Estimation_: In this section, you will estimate the TDOA between pairs of microphones using the provided recordings.\n",
"2. _Localization_: In this section, you will use the estimated TDOAs to locate KITT within the field.\n",
"3. _Deployment_: In this section, you will adapt the code to work on the real car."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Pre-recorded Data\n",
"\n",
"To get you started, we provide you with 7 recordings at known locations and a reference recording taken close to one of the microphones. These can be used to develop and test your algorithms. The recordings are at locations randomly distributed across the field, as follows:\n",
"\n",
"| Recording Index | x [cm] | y [cm] |\n",
"|-----------------|------------|------------|\n",
"| 0 | 64 | 40 |\n",
"| 1 | 82 | 399 |\n",
"| 2 | 109 | 76 |\n",
"| 3 | 143 | 296 |\n",
"| 4 | 150 | 185 |\n",
"| 5 | 178 | 439 |\n",
"| 6 | 232 | 275 |\n",
"\n",
"*Table 1: Locations of the given recordings (in cm)*\n",
"\n",
"The x and y axes of the field are defined as follows, where the numbers refer to the microphone index:\n",
"\n",
"<img src=\"pictures/axisdef.png\" alt=\"Field axis and microphone index definition\" width=\"250px\">\n",
"\n",
"*Figure: Field axis and microphone index definition*\n",
"\n",
"You can assume these positions for the microphones (note the different height of microphone 5, and note that these recordings from 2023 did not use the same locations of the microphones on your field this year, so ensure that your code can easily use a table with other locations):\n",
"\n",
"| Microphone | x [cm] | y [cm] | z [cm] |\n",
"|------------|------------|---------|---------|\n",
"| 1 | 0 | 0 | 50 |\n",
"| 2 | 0 | 460 | 50 |\n",
"| 3 | 460 | 460 | 50 |\n",
"| 4 | 460 | 0 | 50 |\n",
"| 5 | 0 | 230 | 80 |\n",
"\n",
"*Table 2: Locations of the microphones (in cm)*\n",
"\n",
"### Loading and Plotting the Provided Recordings\n",
"\n",
"The code below helps you load and plot the first of the 7 audio signals provided. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Coordinates of the recordings\n",
"record_x = [64, 82, 109, 143, 150, 178, 232]\n",
"record_y = [40, 399, 76, 296, 185, 439, 275]\n",
"\n",
"# List to store filenames\n",
"filenames = []\n",
"\n",
"# Generate filenames based on coordinates\n",
"for i in range(len(record_x)):\n",
" real_x = record_x[i]\n",
" real_y = record_y[i]\n",
" filenames.append(f\"Files/Student Recordings/record_x{real_x}_y{real_y}.wav\")\n",
"\n",
"# Load the first recording\n",
"Fs, recording = wavfile.read(filenames[0])\n",
"\n",
"# Plot the first channel of the first recording\n",
"plt.plot(recording[:, 0])\n",
"plt.title(f\"Recording at x={record_x[0]} cm, y={record_y[0]} cm, Microphone 1\")\n",
"plt.xlabel(\"Sample Index\")\n",
"plt.ylabel(\"Amplitude\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Play around with the recordings and try to understand the data you are working with. Try the following:\n",
"- Load and plot another recording.\n",
"- Zoom in on one of the pulses.\n",
"- Of one pulse, overlay the different microphones. Can you spot the time differences? Can you determine in which corner of the field the recording was made?"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Plot the second recording\n",
"Fs, recording = wavfile.read(filenames[1])\n",
"\n",
"plt.plot(recording[:, 0])\n",
"plt.title(f\"Recording at x={record_x[0]} cm, y={record_y[0]} cm, Microphone 2\")\n",
"plt.xlabel(\"Sample Index\")\n",
"plt.ylabel(\"Amplitude\")\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Zoom in on one of the pulses\n",
"plt.plot(recording[202000:209000, 0])\n",
"plt.title(\"Zoomed in on a pulse\")\n",
"plt.xlabel(\"Sample Index\")\n",
"plt.ylabel(\"Amplitude\")\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Of one pulse, overlay the different microphones.\n",
"plt.plot(recording[202000:209000, 0], alpha=0.5)\n",
"plt.plot(recording[202000:209000, 1], alpha=0.5)\n",
"plt.title(\"Zoomed in on a pulse, first 2 microphones\")\n",
"plt.xlabel(\"Sample Index\")\n",
"plt.ylabel(\"Amplitude\")\n",
"plt.legend([\"Microphone 1\", \"Microphone 2\"])\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 74,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Hints:*\n",
"- You can select the recording by changing the index in: ```Fs, recording = wavfile.read(filenames[index])```.\n",
"- You can select the microphones by changing the index in: ```recording[:, mic]```.\n",
"- These recordings are sampled at 44.1 kHz."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Background Knowledge\n",
"\n",
"In the first week courselab Assignments, you developed algorithms for channel estimation using signals received at two microphones. In this module, we extend this to 5 microphones and use this to locate the car.\n",
"\n",
"### Channel Estimation\n",
"\n",
"The channel estimation problem is as follows:\n",
"\n",
"Suppose we transmit a known signal \\( x[n] \\) over a communication channel and measure the received signal \\( y[n] \\). The channel acts as a filter, which we will assume to be linear and time-invariant (LTI). Therefore, the received signal is a convolution of the transmitted signal by the channel impulse response \\( h[n] \\):\n",
"\n",
"$$\n",
"y[n] = x[n] * h[n]\n",
"$$\n",
"\n",
"Knowing the transmitted signal $x[n]$, can we recover the impulse response of the communication channel $h[n]$ from $y[n]$? This is essentially an inversion problem.\n",
"\n",
"#### Channel Estimation Algorithms\n",
"\n",
"There are several methods for channel estimation:\n",
"\n",
"1. **Deconvolution in Time Domain (ch1):** Involves matrix inversion. It is computationally complex and requires lots of memory.\n",
"\n",
"2. **Matched Filter (ch2):** Avoids matrix inversion by computing the cross-correlation of $y[n]$ with $x[n]$. The channel estimate is given by:\n",
"\n",
"$$\n",
"\\hat{h}[n] = \\frac{1}{\\alpha} \\, y[n] * x[-n]\n",
"$$\n",
"\n",
"This method depends heavily on having a reference signal with good autocorrelation properties: $r[n] = x[n] * x[-n]$ should resemble a delta-spike. The normalization constant $\\alpha$ can be set to $\\alpha = r[0] = \\|x\\|^2$.\n",
"\n",
"3. **Deconvolution in Frequency Domain (ch3):** Involves using the Fast Fourier Transform (FFT). Convolution in time becomes multiplication in frequency, so deconvolution becomes division in the frequency domain:\n",
"\n",
"$$\n",
"\\hat{H}[k] = \\frac{Y[k]}{X[k]}\n",
"$$\n",
"\n",
"Then, the channel impulse response $ \\hat{h}[n] $ is obtained by taking the inverse FFT of $ \\hat{H}[k] $.\n",
"\n",
"This method is efficient but requires handling divisions by zero or very small values, which can be achieved by thresholding. Apart from edge effects, the result should be identical to that of **ch1**.\n",
"\n",
"**Note:** We recommend using method **ch3** (Deconvolution in Frequency Domain) for this module.\n",
"\n",
"### Time Difference of Arrival (TDOA)\n",
"\n",
"After estimating the channel impulse response for each microphone, we can detect the first incoming path (the direct path from KITT to the microphone). This corresponds to the propagation delay of the car beacon to each microphone. However, since we do not know the exact transmission time, we can only obtain relative propagation delays. By taking the differences of these delays between pairs of microphones, we eliminate the unknown transmission time, resulting in Time Difference of Arrival (TDOA) measurements.\n",
"\n",
"These TDOA measurements can be used to estimate the position of KITT in the field.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1: Implementing Channel Estimation\n",
"\n",
"Using the provided reference signal and your channel estimation algorithm (ch3: deconvolution in the frequency domain), deconvolve the recordings to obtain the channel impulse response for each microphone.\n",
"\n",
"#### Steps:\n",
"\n",
"1. **Load and Normalize the Recorded and Reference Signal:**\n",
"\n",
" - The reference signal is provided as a WAV file recorded close to one of the microphones.\n",
" - Normalize the signals to have a maximum amplitude of 1. This is important because the amplitude of the received signals may vary depending on the microphone.\n",
"\n",
"2. **Segment the Signals:**\n",
"\n",
" - Manually extract a single pulse from the reference signal and the recordings. (Later, you will have to automate the segmentation of the recordings.)\n",
"\n",
"3. **Implement the Channel Estimation Algorithm:**\n",
"\n",
" - Use ch3 to estimate the channel impulse response of each of the microphone signals.\n",
" - Be sure to handle divisions by zero or very small values in the frequency domain by applying a threshold.\n",
"\n",
"**Notes and Hints:**\n",
"\n",
"- **Normalization:** You can normalize a signal by dividing it by its maximum absolute value:\n",
"\n",
" ```python\n",
" ref_signal = ref_signal / np.max(np.abs(ref_signal))\n",
" ```\n",
"\n",
"- **FFT Length:** When performing convolution or deconvolution via FFT, ensure that the FFT length is sufficient to avoid \"circular convolution\" effects (you will learn about this in EE3S1). Calculate `N = len(ref_pulse) + len(rec_pulse) - 1`, and zero-pad both ref_pulse and rec_pulse to length $N$ before doing the FFT. This ensures that circular convolution becomes the familiar linear convolution.\n",
"\n",
"- **Thresholding:** Adjust the threshold value based on the magnitude of the FFT of the reference signal. Experiment with different threshold values to find one that works well, starting with a small value (e.g., 1e-3)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# Load and normalize the reference signal\n",
"Fs_ref, ref_signal = wavfile.read(\"Files/Student Recordings/reference.wav\")\n",
"ref_signal = ref_signal[221000:222500, 0] # Use only one channel\n",
"# TODO: Normalize the reference signal\n",
"\n",
"# Plot the reference signal\n",
"plt.figure()\n",
"plt.plot(ref_signal)\n",
"plt.title(\"Reference Signal\")\n",
"plt.xlabel(\"Sample Index\")\n",
"plt.ylabel(\"Amplitude\")\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Load a recorded signal\n",
"# TODO: Select one channel\n",
"# TODO: Normalize the recording\n",
"\n",
"# Plot the recorded signal\n",
"plt.figure()\n",
"plt.plot(recording)\n",
"plt.title(\"Recorded Signal\")\n",
"plt.xlabel(\"Sample Index\")\n",
"plt.ylabel(\"Amplitude\")\n",
"plt.show()\n",
"\n",
"# TODO: Segment the recording to extract one pulse\n",
"\n",
"# Plot the segmented microphone signal\n",
"plt.figure()\n",
"plt.plot(recording_pulse)\n",
"plt.title(\"Segmented Signal\")\n",
"plt.xlabel(\"Sample Index\")\n",
"plt.ylabel(\"Amplitude\")\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Implement the channel estimation algorithm (deconvolution in frequency domain)\n",
"def channel3(ref_pulse, rec_pulse):\n",
" pass\n",
"\n",
"# Estimate the channel impulse response\n",
"\n",
"h_est = channel3(ref_signal, recording_pulse)\n",
"\n",
"# Plot the estimated channel impulse response\n",
"plt.figure()\n",
"plt.plot(h_est)\n",
"plt.title(\"Estimated Channel Impulse Response\")\n",
"plt.xlabel(\"Sample Index\")\n",
"plt.ylabel(\"Amplitude\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Inspect your plots and determine if the impulse response looks realistic (properly zoom in). Do you see a clear peak, at the beginning of the response? "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2: Determining Time of Arrivals\n",
"The goal is to get the time differences of arrival (TDOA) between each pair of microphones. As an intermediate step, we will determine the time of arrivals (TOAs) for each microphone, relative to the reference signal. From the peaks in the channel impulse responses, determine the TOAs and store these in a table.\n",
"\n",
"#### Steps:\n",
"\n",
"1. **Identify the Peaks in the Channel Impulse Responses:**\n",
"\n",
" - The first significant peak corresponds to the direct path from KITT to the microphone.\n",
"\n",
"2. **Calculate the TOAs:**\n",
"\n",
" - Convert the sample indices to time values using the sampling frequency.\n",
"\n",
"3. **Store the TOAs:**\n",
"\n",
" - Collect the TOAs for each microphone and store them in a structured format (e.g., a dictionary or table).\n",
"\n",
"**Notes and Hints:**\n",
"\n",
"- The index of the maximum value in the channel impulse response can be found using:\n",
"\n",
" ```python\n",
" peak_index = np.argmax(np.abs(h_est))\n",
" ```\n",
"\n",
"- Remember that the absolute time of transmission is unknown, so the TOAs are relative to the arbitrary start of your recording.\n",
"\n",
"- If you did not tightly crop the start of your reference signal, a TOA could be negative (the microphone signal appears to arrive earlier than the reference signal); in this case, due to the periodic nature of the IFFT, the peak of the impulse response could be at the far end of the estimated $\\hat{h}[n]$. If in your `ch3` code you chopped the estimate to a shorter length $L$, you might have chopped off this peak and see only noise."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# TODO: Identify the peak in the channel impulse response\n",
"\n",
"# TODO: Calculate the TOA (in seconds)\n",
"\n",
"# TODO:Store the TOA for the microphone\n",
"\n",
"# TODO: Print the TOA"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once you have working code for a single microphone, apply your processing to all 5 microphones. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3: Pulse Segmentation\n",
"\n",
"Uptil now, you have manually selected a single pulse from the recordings. In practice, you will need to automate this process. Implement a pulse detection algorithm to automatically segment the pulses from the recordings. We leave this part open to you, but we provide some hints below. (You could postpone this step until after implementing step 4.)\n",
"\n",
"**Notes and Hints:**\n",
"\n",
"- **Pulse Detection:**\n",
"\n",
" - `find_peaks` from `scipy.signal` can be used to detect peaks in the envelope of the signal. It has many options to help you define what is a useful peak.\n",
" - Adjust the `height` and `distance` parameters based on the characteristics of your signal.\n",
"\n",
"- **Outlier Rejection:**\n",
"\n",
" - You may need to reject some detected pulses that give a very different TOA compared to the other segments.\n",
"\n",
"- **Single Pulse Importance:**\n",
"\n",
" - You may be tempted to compute the channel response for the whole recording. However, the computational complexity of this approach is high. Instead, focus on a single pulse. Also, you might be tempted to produce more accurate results by averaging over several pulses. But remember that this is not really useful in your final application, if you want to do localization on a driving car (in that case, you want to use only the last, most recent pulse)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Develop a function to detect the start indices of pulses in the recording\n",
"# Takes signal and number of pulses in the recordings as input and returns a list of start indices\n",
"def detect_pulses(signal, num_pulses):\n",
" return peaks\n",
"\n",
"# TODO: Load the recording\n",
"\n",
"# Detect pulses in the recorded signal\n",
"pulse_starts = detect_pulses(recording_channel, num_pulses)\n",
"\n",
"plt.plot(recording_channel)\n",
"plt.plot(pulse_starts, recording_channel[pulse_starts], 'x')\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Develop a function that takes in a recording and returns the TOA for each microphone\n",
"def estimate_TOAs(recording, num_pulses):\n",
" TOAs = {}\n",
" return TOAs\n",
"\n",
"Fs, recording = wavfile.read(filenames[1])\n",
"print(estimate_TOAs(recording, 5))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 4: Calculating TDOA Between Microphone Pairs\n",
"\n",
"Complete the TDOA function to calculate the time difference of arrival between pairs of microphones.\n",
"\n",
"#### Steps:\n",
"\n",
"1. **Calculate TDOA Between Two Microphones:**\n",
"\n",
" - For each pair of microphones, calculate the difference in TOAs.\n",
"\n",
"2. **Convert TDOA to Distance Difference:**\n",
"\n",
" - Use the speed of sound (approximately 343 m/s) to convert TDOA to distance difference.\n",
"\n",
"3. **Store and Use TDOA Values:**\n",
"\n",
" - Collect TDOA values for all relevant microphone pairs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"# TODO: Implement a function to calculate TDOA between two microphones\n",
"def calculate_tdoa(TOA1, TOA2):\n",
" pass\n",
" return tdoa\n",
"\n",
"# Load the TOAs for each microphone\n",
"Fs, recording = wavfile.read(filenames[1])\n",
"TOAs = estimate_TOAs(recording, 5)\n",
"\n",
"# TODO: Calculate TDOA between Mic1 and other microphones\n",
"\n",
"# Print the results\n",
"print(f\"TDOA between Mic1 and Mic2: {tdoa_12} seconds\")\n",
"\n",
"print(f\"TDOA between Mic1 and Mic3: {tdoa_13} seconds\")\n",
"\n",
"print(f\"TDOA between Mic1 and Mic4: {tdoa_14} seconds\")\n",
"\n",
"print(f\"TDOA between Mic1 and Mic5: {tdoa_15} seconds\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Sanity Check\n",
"The accuracy of the TDOA measurements is crucial for the localization accuracy. You can test the accuracy of your TDOA estimation by calculating the TDOA based on the known recording positions and comparing it with the estimated TDOA.\n",
"\n",
"Develop a test function using Pythagoras’ theorem that takes an $(x, y)$ position as input and calculates the theoretical TDOAs you would observe from this position for all microphone pairs. This involves calculating the distance from the given point to each microphone.\n",
"\n",
"This will help you:\n",
"\n",
"- Debug your TDOA function.\n",
"- Verify your position estimation in the next steps."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Microphone positions (x, y, z) in meters\n",
"microphone_positions = {\n",
" 1: np.array([0.0, 0.0, 0.50]),\n",
" 2: np.array([0.0, 4.60, 0.50]),\n",
" 3: np.array([4.60, 4.60, 0.50]),\n",
" 4: np.array([4.60, 0.0, 0.50]),\n",
" 5: np.array([0.0, 2.30, 0.80])\n",
"}\n",
"\n",
"# Function to calculate theoretical TDOAs from a given (x, y) position\n",
"def calculate_theoretical_tdoas(car_position, mic_positions):\n",
" # Car's position (x, y, z)\n",
" x_car, y_car = car_position\n",
" z_car = 0.0 # Assuming car's beacon is at z = 0\n",
"\n",
" # Calculate distances and TOAs to each microphone\n",
" toas = {}\n",
" for mic_id, mic_pos in mic_positions.items():\n",
" distance = np.sqrt((mic_pos[0] - x_car)**2 + (mic_pos[1] - y_car)**2 + (mic_pos[2] - z_car)**2)\n",
" toas[mic_id] = distance\n",
"\n",
" # Calculate TDOAs between microphone pairs\n",
" tdoa_dict = {}\n",
" mic_ids = list(mic_positions.keys())\n",
" for i in range(len(mic_ids)):\n",
" for j in range(i+1, len(mic_ids)):\n",
" mic_i = mic_ids[i]\n",
" mic_j = mic_ids[j]\n",
" tdoa = toas[mic_j] - toas[mic_i]\n",
" tdoa_dict[(mic_i, mic_j)] = tdoa\n",
"\n",
" return tdoa_dict\n",
"\n",
"# Example usage:\n",
"# Car position in meters\n",
"car_position = np.array([1.43, 2.96]) # Replace with desired (x, y) position\n",
"\n",
"# Calculate theoretical TDOAs\n",
"theoretical_tdoas = calculate_theoretical_tdoas(car_position, microphone_positions)\n",
"\n",
"# Print the TDOA values\n",
"for mic_pair, tdoa in theoretical_tdoas.items():\n",
" print(f\"TDOA between Mic{mic_pair[0]} and Mic{mic_pair[1]}: {tdoa:.6f} seconds\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Localization Using TDOA Information\n",
"\n",
"Now we arrive at the main question studied in this module: **How can we locate the car using the TDOA estimates?** With 5 microphones, we can compute the TDOAs between all pairs of microphones and obtain 10 TDOA measurements. Next, you need an algorithm to convert these TDOA measurements into the $(x, y)$ location of the car.\n",
"\n",
"We offer two approaches for this:\n",
"\n",
"1. **Linear Algebra Approach:** Study [Appendix C](appendix/Appendix_C.ipynb), which shows a basic algorithm to solve for $(x, y)$ using linear algebra. This algorithm is sub-optimal but should be relatively fast, and is a nice illustration of the use of linear algebra (for those who appreciate this).\n",
"\n",
"2. **Grid Search Method:** Perform a grid search over possible $(x, y)$ positions and evaluate which position best matches the estimated TDOAs. This method can be accurate but may be slower depending on implementation. Consider e.g. iterative grid refinement.\n",
"\n",
"Implement one of these methods inside the coordintate_2d function.\n",
"\n",
"Although we compute the $(x, y)$ position, in reality, the world is 3D, and the microphones have a certain height above the height of the audio beacon (we define the beacon to sit at $z=0$). The algorithm in Appendix C does not take this height into account. However, this extension is straightforward and you should add it."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"# Implement a function to estimate the 2D coordinates of the car\n",
"def coordinate_2d(D12, D13, D14, D15):\n",
" xyMic = np.array([[0, 0, 50], [0, 460, 50], [460, 460, 50], [460, 0, 50], [0, 230, 80]])\n",
" return x, y"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Localization Class\n",
"\n",
"Finally, it is time to set up a processing pipeline by putting everything together. You will develop a `Localization` class that takes a 5-channel recording as input and returns the $(x, y)$ coordinates of the car.\n",
"\n",
"### Suggested Class Structure\n",
"\n",
"**Notes:**\n",
"\n",
"- **Modularity:** Separates different stages of processing for clarity and reusability.\n",
"- **Extensibility:** You can add more methods or attributes as needed.\n",
"- **Implementation:** You need to implement the `estimate_tdoas` method based on your earlier code."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"\n",
"from scipy.io import wavfile\n",
"import logging\n",
"import scipy.signal as signal\n",
"from scipy.fft import fft, ifft\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"class Localization:\n",
" \"\"\"\n",
" The Localization class processes multi-microphone audio recordings to estimate the position of a sound source (e.g., KITT) using Time Difference of Arrival (TDOA) measurements.\n",
"\n",
" Attributes:\n",
" - recording (numpy.ndarray): A 2D array containing the recordings from multiple microphones. Shape is (num_samples, num_mics).\n",
" - refSignal (numpy.ndarray): The reference signal used for channel estimation.\n",
" - num_pulses (int): The number of pulses present in the recording.\n",
" - Fs (int): The sampling frequency of the recordings.\n",
" - mic_positions (numpy.ndarray): Array containing the positions of the microphones in centimeters.\n",
" - localizations (tuple): Estimated (x, y) position of the sound source.\n",
" \"\"\"\n",
" def __init__(self, recording, refSignal, num_pulses, Fs):\n",
" \"\"\"\n",
" Initialize the Localization object with the given recordings and parameters.\n",
"\n",
" Parameters:\n",
" - recording (numpy.ndarray): A 2D numpy array of shape (num_samples, num_mics) containing the recordings from multiple microphones. Each column corresponds to a microphone.\n",
" - refSignal (numpy.ndarray): A 1D numpy array containing the reference signal used for channel estimation.\n",
" - num_pulses (int): The number of pulses in the recording.\n",
" - Fs (int): The sampling frequency of the recordings in Hz.\n",
" \"\"\"\n",
"\n",
" def localization(self):\n",
" \"\"\"\n",
" Process the recordings to estimate the 2D position of the sound source.\n",
"\n",
" This method performs the following steps:\n",
" - Segments the recordings into individual pulses based on the number of pulses.\n",
" - For each pulse:\n",
" - Extracts the pulse segment from the recording.\n",
" - Calculates the Time Difference of Arrival (TDOA) between microphone pairs using the extracted pulse.\n",
" - Averages the TDOA measurements across all pulses.\n",
" - Converts the averaged TDOAs to distance differences.\n",
" - Estimates the 2D coordinates of the sound source using the TDOA measurements.\n",
"\n",
" Returns:\n",
" - x_car (float): Estimated x-coordinate of the sound source in centimeters.\n",
" - y_car (float): Estimated y-coordinate of the sound source in centimeters.\n",
" \"\"\"\n",
" \n",
" return x_car, y_car\n",
"\n",
" def coordinate_2d(self, D12, D13, D14):\n",
" \"\"\"\n",
" Estimate the 2D coordinates of the sound source based on TDOA measurements.\n",
"\n",
" This method constructs a system of linear equations using the TDOA measurements and microphone positions and solves for the (x, y) coordinates of the sound source.\n",
"\n",
" Parameters:\n",
" - D12 (float): Distance difference between Microphone 1 and Microphone 2 in centimeters.\n",
" - D13 (float): Distance difference between Microphone 1 and Microphone 3 in centimeters.\n",
" - D14 (float): Distance difference between Microphone 1 and Microphone 4 in centimeters.\n",
"\n",
" Returns:\n",
" - position (numpy.ndarray): A 1D array containing the estimated x and y coordinates [x, y] in centimeters.\n",
" \"\"\"\n",
"\n",
" return x, y # Return the estimated (x, y) coordinates\n",
" \n",
" def previously_implemented_methods(self):\n",
" # Copy paste the previously implemented methods\n",
" pass\n",
"\n",
"if __name__ == \"__main__\":\n",
" # TODO: Load the recording\n",
"\n",
" # TODO: Load the reference signal\n",
"\n",
" # TODO: Initialize the Localization object\n",
" \n",
" # TODO: Get the localized position\n",
"\n",
" print(f\"Estimated position: x={x_car}, y={y_car}\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Optional Extensions\n",
"\n",
"If you finish the basic assignment quickly and want to challenge yourself further, try adding additional functionality to your program. For example:\n",
"\n",
"### 1. Accounting for Height Differences\n",
"\n",
"The current set of linear equations does not consider the height difference between the microphones and the car. This leads to slight inaccuracies, especially when the car is close to a microphone.\n",
"\n",
"**Extension Task:**\n",
"\n",
"- Augment the equations to include the $z$ variable (height of the car).\n",
"- Use the known value of $z_{\\text{car}} = 0$ to eliminate the variable or adjust the equations accordingly.\n",
"\n",
"### 2. Implementing Advanced Localization Algorithms\n",
"\n",
"The provided method is simple but may be unreliable in certain situations (e.g., when the distances to two microphones are equal).\n",
"\n",
"**Extension Task:**\n",
"\n",
"- Research and implement more advanced algorithms from literature, such as:\n",
"\n",
" - Stephen Bancroft, “An algebraic solution of the GPS equations”, IEEE Transactions on Aerospace and Electronic Systems, vol.21, no.7, pp.56-59, January 1985.\n",
" - Amir Beck, Petre Stoica, and Jian Li, “Exact and Approximate Solutions of Source Localization Problems”, IEEE Transactions on Signal Processing, vol.56, no.5, pp. 1770-1778, May 2008.\n",
"\n",
"### 3. Grid Search Method\n",
"\n",
"Instead of the linear algebra approach, implement a grid search where the room is partitioned into a dense grid of possible positions, and each location is evaluated against the TDOA data to find the best fit.\n",
"\n",
"**Extension Task:**\n",
"\n",
"- Perform a coarse grid search with larger steps (e.g., 10 cm).\n",
"- Refine the search in regions with the best matches.\n",
"\n",
"**Hints:**\n",
"\n",
"- Calculate the theoretical TDOAs for each grid point.\n",
"- Compute an error metric (e.g., sum of squared differences) between theoretical and measured TDOAs.\n",
"- Select the grid point with the minimum error as the estimated position.\n",
"\n",
"### 4. Robustness\n",
"\n",
"Along with the estimated $(x,y)$ location, your code could also return a parameter that indicates the reliability of that position. For this you could use the equation error (of the estimated TDOA vs the theoretical TDOA expected for this $(x,y)$), which is basically the same as the value of the cost function in the grid search. Subsequent modules that use your $(x,y)$ estimate could then judge whether to use or reject this position."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Assessment and Reporting\n",
"\n",
"### Mid-term Assessment and Report\n",
"\n",
"Ultimately in Week 4, you will showcase the functionality of your localization script to your assigned TA. You should demonstrate proper localization of the car on the 3 recordings with unknown coordinates.\n",
"\n",
"After you pass this assessment, you are ready to document your results in your midterm report. A detailed report is required, covering:\n",
"\n",
"- **Approach:** Describe the methods and algorithms used.\n",
"- **Implementation:** Explain how you implemented the algorithms.\n",
"- **Testing:** Show results from tests on simulated and provided test data.\n",
"- **Results:** Present localization results, error analysis, and accuracy.\n",
"- **Conclusion:** Summarize the expected accuracy and reliability of your localization method.\n",
"\n",
"Please review the guidelines in Chapter 7 for more information.\n",
"\n",
"### After the Midterm: Reference Signal and Integration\n",
"\n",
"If you have completed this module successfully, you can start integrating and estimating the car’s location from your own microphone measurements.\n",
"\n",
"#### Reference Signal Selection\n",
"\n",
"Using a good beacon signal is important, because it determines the quality of your location estimates. Since the time in IP3 is limited, we will give a few general hints, but note that the topic could be explored in much more detail.\n",
"\n",
"In your ch3 algorithm (deconvolution in frequency domain), you have seen that we omit all frequencies where the beacon signal amplitude spectrum $|X(F)|$ is small, because we divide by $X(F)$ and this would blow up the noise. For best results, we want $X(F)$ to be large for all frequencies. Therefore, ideally, we have a flat power spectrum. For random signals, this corresponds to the use of \"white noise\". \n",
"\n",
"**Beacon code** Our beacon code consists of 32 bits, each 0 or 1. White noise corresponds to a beacon code for which the autocorrelation resembles a delta spike. You could generate a bit code using 'rand', and then round to 0 or 1. A further consideration is that the code should start and end with '1', or else you are using in fact a code that is shorter than 32 bits. \n",
"\n",
"After generating a code, check its autocorrelation function, $x[n]\\ast x[-n]$. \n",
"You want a strong peak for the 0-lag of the autocorrelation but as low as possible for any other lag. You can also check its spectral properties.\n",
"*Hint:* Randomly generated codes are suitable for our purposes. You could try some optimal codes (check communication theory literature for “gold codes”), but don't waste too much time in trying to find the best code.\n",
"\n",
"**Carrier frequency and bit frequency** The carrier frequency defines the \"pitch\" of the transmitted signal. Since we use audio equipment which is optimized for human listening, it is probably better to use carriers below 10 kHz. \n",
"\n",
"The bit frequency determines the bandwidth of the transmitted signal. Since we want to probe the channel on as many different frequencies as possible, we would select a large bit frequency (but it can't be larger than the carrier frequency). The hardware implementation seems to limit the total output power. If we use a wider bandwidth, the energy per hertz is reduced. Thus, there is a trade-off. You can try to find good values experimentally, i.e. try a range of bit frequencies in steps of 1000 Hz, and see which setting gives most accurate results.\n",
"\n",
"_Note:_ Last year it appeared the implementation of the firmware on the car has a bug which prevents modifying the beacon parameters? Let us know if you experience this.\n",
"\n",
"A perfect repetition count is not yet required; but you need to make sure that the full code can be transmitted and recorded on all microphones within the same recording window.\n",
"\n",
"**Field Testing:** A theoretical signal may not work well in practice. Use your field test time well. Consider recording a bunch of different audio beacon settings and analyzing them later.\n",
"\n",
"### Integration Assignment\n",
"\n",
"#### **Make Test Recording**\n",
"\n",
"- Record the signal transmitted over KITT’s beacon with your selected parameters.\n",
"- Place the microphone close to KITT’s beacon to get a clean recording.\n",
"- Avoid clipping by adjusting recording levels.\n",
"\n",
"#### **Create Reference Signal**\n",
"\n",
"- Load the recording into Python.\n",
"- Clean it by removing zero intervals and extracting a single pulse.\n",
"- Use this as your reference signal in deconvolution.\n",
"\n",
"#### **Test Performance**\n",
"\n",
"- Place KITT at a known location (e.g., the center of the field).\n",
"- Make recordings and run your localization algorithm.\n",
"- Evaluate the accuracy.\n",
"\n",
"#### **Integrate with Recording Code**\n",
"\n",
"- Combine your localization algorithm with the recording code.\n",
"- Address any issues such as blocking recording.\n",
"- Aim to locate KITT in real-time while it is moving.\n",
"\n",
"#### **Refinements Needed for a Driving Car**\n",
"\n",
"- Add timestamp information to your location estimates.\n",
"- Segment your recording and take only the last (most recent) pulse.\n",
"- Consider the delays between recording, processing, and KITT's movement. Figure out how long ago was the most recent pulse.\n",
"- Use a high repetition rate and short recording lengths.\n",
"- Check out the difference between \"blocking\" and \"non-blocking\" recordings. Perhaps you will need multi-threading such that the localization can run in parallel with the control.\n",
"\n",
"### Deliverable (Final Report)\n",
"\n",
"Show your selected beacon parameters and comment on the resulting performance.\n",
"\n",
"- **Accuracy:** How accurate is your localization algorithm?\n",
"- **Real-time Operation:** Are you able to drive and locate KITT simultaneously in real-time?\n",
"- **Discussion:** Address any challenges faced and how you overcame them.\n",
"\n",
"**Note to Students:**\n",
"\n",
"- Remember to document your code thoroughly.\n",
"- Include explanations for your choices and any assumptions made.\n",
"- Test your algorithms extensively with both simulated and real data.\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "booktestenv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.4"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

614
6_Module_4.ipynb Normal file
View File

@ -0,0 +1,614 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](pictures/header.png)\n",
"\n",
"[Table of Contents](0_Table_of_Contents.ipynb)\n",
"\n",
"# Module 4: Car Model\n",
"**Contents:**\n",
"* [Velocity Model](#velocity-model)\n",
"* [Steering Model](#steering-model)\n",
"* [Building a Unified Car Model](#building-a-unified-car-model)\n",
"* [Assessment](#assessment)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import necessary libraries\n",
"import time\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# Uncomment one of the following lines depending on your setup\n",
"\n",
"# If you are using the real car, uncomment the next line and comment the simulator line\n",
"# from serial import Serial\n",
"\n",
"# If you are using the simulator, uncomment the next line and comment the real car line\n",
"from KITT_Simulator.serial_simulator import Serial\n",
"\n",
"# Note: After changing the import statement, you need to restart the kernel for changes to take effect."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this Module, we give some basic ingredients to model KITT's behavior. The goal is to use this model to predict its\n",
"state (position, orientation, velocity) 1 to 2 seconds ahead of time. This will allow you to drive for some time before you receive a new localization estimate from Module 3.\n",
"\n",
"This module is divided into two parts: a velocity model and a steering model. By the end of the module, you will have created Python code that simulates the car's movement based on a model of its physics, and calibrated by real-world data.\n",
"\n",
"Using the velocity model, you will be able to predict the car's velocity based on the throttle and time. Using the steering model, you will be able to predict the car's orientation based on the steering wheel setting and time. This will enable you to plan what commands to send to the car to make it move to a desired location."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Velocity Model\n",
"\n",
"### Forces Acting on KITT\n",
"\n",
"We start by driving on a straight line, and consider a simple model of longitudinal KITT dynamics,\n",
"described by Newton’s second law, $F=m a$. The car’s motion with mass $m$ is influenced by the following three forces:\n",
"\n",
"- $F_{motor}$: The driving force provided by the motor.\n",
"- $F_{friction}$: The frictional force opposing the motion, which is proportional to the velocity, $F_{friction} = -b \\, v$, where $b$ is the friction coefficient.\n",
"- $F_{{air}}$: The air resistance, which is proportional to the square of the velocity, $F_{{air}} = -c \\, v^2$, where $c$ is the air resistance coefficient.\n",
"\n",
"Assuming $v$ is positive, the net resulting force acting on the car can be expressed as:\n",
"\n",
"$$ F_{{res}} = F_{{motor}} - b v - c v^2 $$\n",
"\n",
"\n",
" <img src=\"pictures/KITTwind.jpg\" alt=\"KITTwind\" width=\"350px\">\n",
"\n",
" *Indicate the forces and their directions.* \n",
"\n",
"Note that another force acting on the car could be the braking force $F_b$. \n",
"Unfortunately, KITT does not have a brake! You can stop by letting KITT roll to standstill, or for a\n",
"short period apply a negative force $F_{motor}$. The difference with a real brake is that if you apply $F_{motor}$ for\n",
"too long, or were already stopped, the car will drive backwards.\n",
"\n",
"### Newton's Second Law of Motion\n",
"\n",
"According to Newton's second law, the net force acting on the car is equal to the mass of the car times its acceleration:\n",
"\n",
"$$ F_{{res}} = m a = m \\frac{{\\mathrm d}v}{{\\mathrm d}t} $$\n",
"\n",
"Substituting the expression for $F_{res}$ gives\n",
"\n",
"$$ m \\frac{{\\mathrm d}v}{{\\mathrm d}t} = F_{{motor}} - b v - c v^2 $$\n",
"\n",
"### Velocity as a Function of Time\n",
"\n",
"To find the velocity as a function of time, we need to solve the differential equation:\n",
"\n",
"$$ \\frac{{\\mathrm d}v}{{\\mathrm d}t} = \\frac{F_{{motor}}}{m} - \\frac{b}{m} v - \\frac{c}{m} v^2 $$\n",
"\n",
"This is a nonlinear differential equation due to the $v^2$ term. However, if air resistance is small compared to friction, we can neglect the $ v^2$ term for a simplified model:\n",
"\n",
"$$ \\frac{{\\mathrm d}v}{{\\mathrm d}t} = \\frac{F_{{motor}}}{m} - \\frac{b}{m} v $$\n",
"\n",
"The solution to this first-order linear differential equation is:\n",
"\n",
"$$ v(t) = \\frac{F_{{motor}}}{b} \\left(1 - e^{-\\frac{b}{m}t}\\right) $$\n",
"\n",
"This is of the form: $v(t) = A( 1 - e^{-t/\\tau})$, where $\\tau=m/b$ is a time constant, similar to the RC-time in an electric circuit, and $A$ is the final (constant) velocity that the car is able to reach.\n",
"\n",
"_Exercise_: From this expression for $v(t)$, derive an expression for the position $x(t)$. You can also extend the solutions to take a nonzero initial velocity $v(0)$ and $x(0)$ into account."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Implementing the Velocity Model\n",
"Now we will use the above equations to calculate the velocity of the car over time for a single value of the motor command $F_{motor}$. In the example simulation below, we assume some random car parameters, but note that these values are plausible but not accurate. Later, you will need to estimate these constants for your own car. For increased accuracy, you might even need to repeat this calibration for multiple battery values. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"# Car parameters, set them to the real values\n",
"# You can use these random values until you measure the real ones\n",
"m = 5.6 # actual mass of the car (kg)\n",
"b = 5.0 # random friction coefficient (N·s/m)\n",
"\n",
"# motor command\n",
"F_motor = 10.0 # random constant motor force (N)\n",
"\n",
"# simulation parameters\n",
"dt = 0.01 # time step (s)\n",
"t_max = 10.0 # maximum simulation time (s)\n",
"t = np.arange(0, t_max, dt) # Time array between 0 and t_max with time step dt\n",
"\n",
"# Velocity array\n",
"v = np.zeros_like(t) # Empty array to store the velocity values\n",
"v[0] = 0.0 # initial velocity (m/s)\n",
"\n",
"# Numerical simulation (Euler's method)\n",
"for i in range(1, len(t)):\n",
" # TODO: Implement the model without air resistance (c=0) straight from the differential equation\n",
" dv = \n",
" v[i] = \n",
"\n",
"# Plotting the results\n",
"plt.figure(figsize=(6, 4))\n",
"plt.plot(t, v, linestyle='--')\n",
"plt.xlabel('Time (s)')\n",
"plt.ylabel('Velocity (m/s)')\n",
"plt.title('Velocity of the RC Car Over Time')\n",
"plt.grid(True)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Calibrating the Velocity Model\n",
"\n",
"In this section, you will try to estimate the parameters $F_{motor}$ and $b$ from measured data. The mass of the car was weighed and is $m=5.6$ kg. To find the other parameters, we will closely examine the velocity equation and write some code to make measurements on the car. This part is less guided so you will have to find some of your own solutions. \n",
"\n",
"Recall that the velocity equation (zero initial conditions) is\n",
"\n",
"$$ v(t) = \\frac{F_{{motor}}}{b} \\left(1 - e^{-\\frac{b}{m}t}\\right) $$\n",
"\n",
"Note that for sufficiently large $t$, we see that the velocity becomes constant: $v(\\infty) = \\frac{F_{{motor}}}{b}$. This is called the terminal velocity.\n",
"\n",
"#### Finding the Friction Coefficient $b$\n",
"\n",
"1. **Measure Terminal Velocity:**\n",
" - Run the car at a motor command setting of interest towards a wall. Measure the velocity as the car accelerates until $v_{\\infty}$ is reached. For this you can use the distance sensors. _Please place one of the bumber boxes in front of the car to stop it from crashing into the wall._ Transform these distance measurements into a velocity plot (remove duplicates and merge the L and R sensor data, as done in Module 2). Plot both position and velocity as function of time.\n",
" \n",
"2. **Initial Acceleration:**\n",
" - At the very beginning of the motion, the car starts from rest, and the initial velocity $v(0) = 0$. At this point, frictional forces are negligible since $v$ is small, and the acceleration ${\\mathrm d}v/{\\mathrm d}t$ can be approximated by\n",
" $$ \\frac{{\\mathrm d}v}{{\\mathrm d}t} \\approx \\frac{F_{{motor}}}{m} $$\n",
" - You can estimate this value from the slope of the $v(t)$ plot near $t=0$. Assuming we know $m$, we find an estimate for $F_{motor}$.\n",
"\n",
"3. **Calculate $b$ Using Terminal Velocity:**\n",
" - With $F_{{motor}}$ known, use the measured terminal velocity $v_\\infty$ to calculate the friction coefficient $b$:\n",
" $$ b = \\frac{F_{{motor}}}{v_{{\\infty}}} $$\n",
"\n",
"The above process is just one approach. Using measured data, it might be less accurate because it requires you to make a velocity plot (taking numerical derivatives magnifies noise) and then even a second derivative (the slope of $v(t)$). Alternatively, you can work with a plot of position vs time, and measure the terminal velocity as the slope of the $x(t)$ plot for larger time. The time constant $\\tau = m/b$ can be determined from a velocity plot by marking the time where the velocity is $1/e \\approx 0.368$ of its final value. \n",
"\n",
"- Once you have estimated your parameters, plot the resulting model curves (for velocity and position) on top of the measured curves. Comment on these plots: how well is the fitting?\n",
"\n",
"- $F_{{motor}}$ depends on the speed setting, and also on the battery status; it will have to be measured again if these values change. For your application, you probably need only one or two speed settings, but you can try to make a calibration table for multiple battery levels.\n",
"\n",
"\n",
"- The value of $b$ should be independent of the speed setting. If you have a calibration table, can you verify this? If it appears constant, you could use an average $b$ for improved accuracy, rather than a separate $b$ for each speeed setting.\n",
"\n",
"#### Finding the response time\n",
"\n",
"As you try to fit the theoretical first order model for $x(t)$ to your measured data, you will probably find that the model is not complete: between giving a speed command and the car responding is a rather large reponse time $t_0$, which delays the theoretical response to $x(t-t_0)$. \n",
"\n",
"Try to estimate $t_0$ and investigate if this parameter allows you to make a better fit. \n",
"\n",
"### Model for driving backwards\n",
"\n",
"You don't always drive at full speed, occasionally you also will want to stop the car. There are 2 options for this: setting the speed to neutral (150) and letting the car roll to standstill, or briefly set the motor speed to drive backwards. The first option is simpler to model because the second one has multiple parameters that you need to set (and they might be dependent on your current speed); however, the second one is probably also much faster and perhaps more accurate.\n",
"\n",
"Choose one of these two methods and create a model for it, with as input the initial velocity $v(0)$, where $t=0$ is the time you set the speed to neutral or decide to drive backwards. What parameters need to be calibrated? For the rest, the implementation of this model is not that different than the model that you already have. \n",
"\n",
"Show plots to show how well your model for $x(t)$ fits measured data.\n",
"In the end, the model needs to tell you how far the car will drive, i.e. $x(\\infty)$.\n",
"\n",
"### Combined model for driving straight\n",
"\n",
"Implement your model into a function, or better still, a `class` object. The function should take as input a driving command and a time duration, and have a loop that integrates the differential equation over small time steps to the requested time. The `class` wrapper around it contains state parameters such as current time, position, and velocity, and update the state to the requested time. \n",
"\n",
"The implementation of the response time might be a bit more tricky: you also need to have the current speed setting as part of the state, and if the newly requested speed setting is different, switch to the new setting at the appropriate moment (integrating using the old setting until that moment).\n",
"\n",
"**Jump ahead to the section below: \"Building a Unified Car Model\",** to see how we envision you can build your model class.\n",
"Here, you can skip the parts that have to do with steering. \n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Mid-term Assessment and Report\n",
"\n",
"Ultimately in Week 5, you will showcase the functionality of your 1D car model to your assigned TA. \n",
"The TA will give you a short series of driving commands (e.g., M165 for 1.5 sec, M150), your model predicts the final position, and this is compared to the actual position of KITT after following the same series of commands.\n",
"\n",
"In the midterm report, present the model, and present test results that shows the accuracy of this model,\n",
"starting from a known state. Summarize in a conclusion: over what time period can you predict the new\n",
"position of the car with an accuracy better than 30 cm?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## After the Mid-term: Steering Model\n",
"\n",
"To model the steering behavior of a 4-wheeled car, often the Ackermann steering model is used. However, we will work with a simplified version that models the steering of a bicycle. \n",
"\n",
"### The Bicycle Steering Geometry\n",
"\n",
"\n",
"Consider the following figure:\n",
"\n",
"<img src=\"pictures/bicycle.png\" alt=\"Bicycle Steering Model\" width=\"600\" />\n",
"\n",
"In this figure, we have\n",
"- $\\theta$: the heading (or orientation) of the car\n",
"- $\\delta$: the steering angle (you determine this by a steering command)\n",
"- $L$: the wheel base\n",
"- $R$: the radius of the turning circle\n",
"- $\\omega = {\\mathrm d}\\theta / {\\mathrm d} t$: rate of angle change (or angular velocity) \n",
"\n",
"We choose to measure $R$ with respect to the rear axle. From the geometry, we can then derive the following two equations:\n",
"- $\\frac{L}{R} = \\tan(\\delta) \\qquad \\Rightarrow \\quad R = \\frac{L}{\\tan(\\delta)}$\n",
"- $\\omega = \\frac{v}{R}$\n",
"\n",
"To verify this equation, consider that we drive with a constant velocity over an entire circle (rotation angle $2\\pi$, circumference $2 \\pi R$, driving time $T$):\n",
"- $\\omega T = 2\\pi \\quad \\Rightarrow \\quad \\omega = \\frac{2\\pi}{T}$\n",
"- $2\\pi R = v T \\quad \\Rightarrow \\quad \\omega = \\frac{2\\pi}{2\\pi R}\\cdot v = \\frac{v}{R}$ \n",
"\n",
"The expression $R = L/\\tan(\\delta)$ can be used to create a calibration table: for various steering commands, we measure the resulting $R$ and compute the corresponding $\\delta$. If we then apply a new (uncalibrated) angle setting, we can interpolate the table to find $\\delta$ and then calculate the corresponding radius $R$.\n",
"\n",
"The expression for $\\omega$ will be useful in our implementation of the car model where we integrate over small time steps $\\Delta t$."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"# Car Parameters\n",
"L = 0.335 # Wheelbase (meters)\n",
"R = 1.5 # Measured turning radius (meters) for some steering command\n",
"\n",
"# TODO: Calculate the steering angle delta based on the bicycle model\n",
"delta = \n",
"\n",
"# Convert to degrees for easier interpretation\n",
"delta_deg = np.degrees(delta)\n",
"\n",
"print(f\"Steering Angle: {delta_deg:.2f} degrees\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Building a Unified Car Model\n",
"\n",
"In this section, you will integrate the velocity model and a steering model into a single, unified Python class. For this you will create all sections step-by-step and later combine them into a single class to re-use in your main code.\n",
"\n",
"### Step 1: Setting Up the Car’s Physical Properties\n",
"\n",
"To begin, we need to define the physical properties of KITT. These include its mass, friction coefficient, and motor force. These properties will be used in our calculations. Later, you add a calibration table to translate a speed setting to the corresponding motor force."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"# TODO: Step 1: Define the physical properties of the RC car\n",
"mass = 5.6 # kg, mass of the car\n",
"wheelbase = 0.335 # m, distance between the front and rear axles\n",
"friction_coefficient = # N·s/m, resistance due to friction\n",
"motor_force = # N, force generated by the motor for certain speed setting (should later be set from a calibration table)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 2: Initializing the Car’s State\n",
"\n",
"Next, we need to define and initialize the state of the car. The state includes the car’s position, velocity, orientation, and the current time. Before the mid-term, we will store the position as a 1D coordinate $x$, and later extend it to a 2D coordinate $[x, y]$. The velocity is a scalar (speed in meters per second), the orientation is an angle (in radians), and time is a scalar (seconds). _In this chapter, the position of the car refers to the center of the rear axle. At some point, you have to translate this to a more useful reference point, i.e. the center of the beacon._"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Step 2: Initialize the car's state\n",
"position = np.array([0.0, 0.0]) # (x, y) position in meters\n",
"velocity = 0.0 # initial speed in m/s\n",
"orientation = 0.0 # initial orientation in radians (0 means facing along the x-axis)\n",
"time = 0.0 # start time in seconds; position, velocity and orientation refer to this time"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 3: Updating the Car’s Velocity\n",
"\n",
"KITT's velocity depends on the force applied by the motor, as well as the friction and air resistance. You have already derived the equation for updating the current velocity $v[i]$:\n",
"\n",
"$$ a[i] = \\frac{F_{{motor}}}{m} - \\frac{b}{m} v[i]$$\n",
"\n",
"so that\n",
"\n",
"$$ v[i+1] = v[i] + a[i] \\cdot \\Delta t$$\n",
"\n",
"We can now implement this equation as a function in Python:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"# TODO: Step 3: Define a function to update the car's velocity\n",
"def update_velocity(velocity, F_motor, mass, friction_coefficient, dt):\n",
" # TODO: Calculate acceleration\n",
" acceleration = \n",
" # TODO: Update velocity\n",
" velocity += \n",
" return velocity"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"### Step 4: Updating the Car’s Orientation\n",
"\n",
"The car’s orientation changes based on the turn radius. The bicycle steering model helps us determine how much the car’s orientation should change given the current velocity and turning radius.\n",
"\n",
"The change in orientation can be calculated as:\n",
"\n",
"$$ \\omega = \\frac{v}{R}$$\n",
"\n",
"$$ \\Delta \\theta = \\omega\\cdot \\Delta t $$\n",
"\n",
"so that\n",
"\n",
"$$ \\theta[i+1] = \\theta[i] + \\Delta\\theta$$"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"# TODO: Step 4: Define a function to update the car's orientation\n",
"def update_orientation(orientation, velocity, turn_radius, dt):\n",
"\n",
" return orientation"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 5: Updating the Car’s Position\n",
"\n",
"The car’s position changes based on its velocity and orientation. Let's store the position in a 2D vector $\\mathbf z$. We can use basic trigonometry to update the position:\n",
"\n",
"\n",
"$$ {\\mathbf z}[i+1] = {\\mathbf z}[i] + v[i] \\cdot\\Delta t \\cdot [\\cos(\\theta[i]), \\sin(\\theta[i])] $$"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"# TODO: Step 5: Define a function to update the car's position\n",
"def update_position(position, velocity, orientation, dt):\n",
" position[0] += # Update x-coordinate\n",
" position[1] += # Update y-coordinate\n",
" return position"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 6: Integrating Everything into a Car Class\n",
"\n",
"Now that you have functions to update each part of the car’s state, it’s convenient to integrate everything into a single class. This class will manage the car’s state and provide an easy interface to simulate its movement."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"### Student Version ###\n",
"# TODO: Step 6: Create the KITT_Dynamics class\n",
"class KITT_Dynamics:\n",
" def __init__(self, mass, wheelbase, friction_coefficient, motor_force):\n",
" # Physical properties of the car\n",
" self.mass = mass\n",
" self.wheelbase = wheelbase\n",
" self.friction_coefficient = friction_coefficient\n",
" self.motor_force = motor_force\n",
" \n",
" # Initial state\n",
" self.position = np.array([0.0, 0.0]) # (x, y) position\n",
" self.velocity = 0.0 # initial velocity in m/s\n",
" self.orientation = 0.0 # initial orientation in radians\n",
" self.time = 0.0 # start time in seconds\n",
" \n",
" def update(self, motor_on, turn_radius, dt):\n",
" # Determine the motor force\n",
" F_motor = self.motor_force if motor_on else 0.0\n",
" \n",
" # Update the car's state\n",
" self.velocity = update_velocity(self.velocity, F_motor, self.mass, self.friction_coefficient, dt)\n",
" self.orientation = update_orientation(self.orientation, self.velocity, turn_radius, dt)\n",
" self.position = update_position(self.position, self.velocity, self.orientation, dt)\n",
" \n",
" # Update time\n",
" self.time += dt\n",
" \n",
" def get_state(self):\n",
" return {\n",
" 'position': self.position,\n",
" 'velocity': self.velocity,\n",
" 'orientation': self.orientation,\n",
" 'time': self.time\n",
" }\n",
" \n",
" @staticmethod\n",
" # TODO: Step 3: Define a function to update the car's velocity\n",
" def update_velocity(velocity, F_motor, mass, friction_coefficient, dt):\n",
" # TODO: Calculate acceleration\n",
" acceleration = \n",
" # TODO: Update velocity\n",
" velocity += \n",
" return velocity\n",
" \n",
" @staticmethod\n",
" # TODO: Step 4: Define a function to update the car's orientation\n",
" def update_orientation(orientation, velocity, turn_radius, dt):\n",
"\n",
" return orientation\n",
" \n",
" @staticmethod\n",
" # TODO: Step 5: Define a function to update the car's position\n",
" def update_position(position, velocity, orientation, dt):\n",
" position[0] += # Update x-coordinate\n",
" position[1] += # Update y-coordinate\n",
" return position"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Step 7: Running the Simulation\n",
"\n",
"With the car model complete, you can now simulate the car’s movement over time. We do this by computing the position of the car in steps of dt, storing the positions in an array, and finally plotting them."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Step 7: Simulate the car's motion\n",
"dt = 0.01 # time step (s)\n",
"simulation_time = 10.0 # total simulation time (s)\n",
"turn_radius = 1.0 # turn radius in meters (positive for right, negative for left)\n",
"\n",
"# Initialize the car model\n",
"car = KITT_Dynamics(mass, wheelbase, track_width, friction_coefficient, motor_force)\n",
"\n",
"# Arrays to store the simulation data\n",
"positions = []\n",
"velocities = []\n",
"orientations = []\n",
"times = []\n",
"\n",
"# Run the simulation\n",
"for t in np.arange(0, simulation_time, dt):\n",
" if t < 5.0:\n",
" car.update(motor_on=True, turn_radius=turn_radius, dt=dt)\n",
" else:\n",
" car.update(motor_on=False, turn_radius=0, dt=dt)\n",
" state = car.get_state()\n",
" \n",
" positions.append(state['position'].copy())\n",
" velocities.append(state['velocity'])\n",
" orientations.append(state['orientation'])\n",
" times.append(state['time'])\n",
" \n",
"# Convert to numpy arrays for easier plotting\n",
"positions = np.array(positions)\n",
"velocities = np.array(velocities)\n",
"orientations = np.array(orientations)\n",
"times = np.array(times)\n",
"\n",
"# Plotting the car's path with color gradient based on time\n",
"plt.figure(figsize=(6, 4))\n",
"sc = plt.scatter(positions[:, 0], positions[:, 1], c=times, cmap='viridis', label=\"Car Path\", s=5)\n",
"plt.xlabel(\"X Position (m)\")\n",
"plt.ylabel(\"Y Position (m)\")\n",
"plt.title(\"Path Over Time\")\n",
"plt.axis('equal')\n",
"plt.grid(True)\n",
"plt.colorbar(sc, label=\"Time (s)\")\n",
"plt.show()\n",
"\n",
"# Plotting the car's velocity over time\n",
"plt.figure(figsize=(6, 4))\n",
"plt.plot(times, velocities, label=\"Velocity (m/s)\")\n",
"plt.xlabel(\"Time (s)\")\n",
"plt.ylabel(\"Velocity (m/s)\")\n",
"plt.title(\"Velocity Over Time\")\n",
"plt.grid(True)\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The scripts shown above are able to create realistic car tracks. However, they need to be extended/adjusted by you to take into account:\n",
"- translation of your speed setting command to a corresponding $F_{{motor}}$ (calibration table)\n",
"- translation of your steering setting command to a corresponding turning circle $R$ (calibration table / interpolation function)\n",
"- translation of the internally tracked position (mid of real axle) to a better reference point (car beacon)\n",
"- nonzero initial conditions\n",
"\n",
"At some point you will probably find that the car behavior is not constant; e.g., the velocity response depends on the battery status and also (slightly) on the steering setting. You could extend your model to take that into account."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "OpenEdVenv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

114
7_Midterm_Report.ipynb Normal file
View File

@ -0,0 +1,114 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](pictures/header.png)\n",
"[Table of Contents](0_Table_of_Contents.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Mid-term report\n",
"\n",
"Now that you know how to drive your car, and hopefully have a working localization module and car model, it is time to document what you did so far in a mid-term report. This report will also form the basis of your final report.\n",
"\n",
"**Report** A mid-term report explaining primarily the locallization and modeling results. Also answer the specific questions asked in the modules. \n",
"*Suggested length:* About 15 to at most 20 pages (excluding the Appendix that lists the Python code).\n",
"The report is prepared as a group. The report is graded and contributes to your final grade.\n",
"\n",
"**Preparation** Completed the preceding modules (sign-off by the TAs).\n",
"\n",
"**Time duration** Two homework sessions.\n",
"\n",
"## Mid-term report guidelines\n",
"\n",
"The mid-term report explains your designs with its techincal details. The primary focus is to report on your results for the localization and modeling modules, but also the distance sensor measurements should be reported. \n",
"\n",
"Aspects on which the report is judged are:\n",
"\n",
"- *Technical content:* Theoretical justification and accuracy of the results, addressing all requested tasks. Bonus for taking the analysis/design beyond the strict scope of the related Module, in particular for introducing innovative solutions; penalty for unacceptable conceptual mistakes or unfinished work.\n",
"\n",
"In IP3, we place particular emphasis on _Proof of your results_ (localization algorithm, car model) using sufficiently extensive testing. If you claim something \"works\", you have to show evidence for this.\n",
"\n",
"- *Quality of the submitted report:* Conformity to the requirements of a scientific report (adequate use of equations, figures, citations, cross-references), readability, layout. \n",
"The format is of that of a technical report, and not that of a homework assignment. Thus, the report should contain the results of the various assignments, embedded in a natural way, as these will motivate your solution. \n",
"\n",
"- *Planning and teamwork* are also to be judged. The report should also have a section that clearly describes these aspects. \n",
"\n",
"The deadline for submission is listed on Brightspace. Submit your report using the corresponding submission folder. Please use a filename that starts with your group number.\n",
"\n",
"The report should be independently readable by a technically skilled committee member who is not familiar with IP3, but you can refer where needed, so don't copy large parts of this manual but summarize the scope in your own words. Note: you cannot refer to something like 'ch3' without explaining briefly what that is.\n",
"\n",
"The modules suggested questions that you can answer in your report ---do this in a natural (self-contained) way. You can also refer to additional literature that you consulted. As mentioned, you should try to be concise, and judge yourself what is important to be included.\n",
"\n",
"If you used AI to generate code or text, then this has to be acknowledged (under the current conventions, which are likely to change in future).\n",
"\n",
"## How to write and structure your report\n",
"\n",
"Here are some general directions on the structure and contents of your report (mid-term and final report). For the mid-term report, the main focus will be on three sections: sensor data, localization, and car model; other chapters such as system integration and conclusions are not yet relevant and can be omitted.\n",
"\n",
"Keep a clear structure and writing style. Make sure you give sufficient factual information (in particular if things don't work). The language should *not* be informal. Keep things concise, ''bla-bla'' is not appreciated. If you make claims such as ``something is the best'', first provide evidence (or at least a motivation, or a reference to literature).\n",
"\n",
"**Cover:** Make sure you specify names, study number, group number, and date.\n",
"\n",
"**Introduction:** First determine who is the reader (in this case, the course coordinator, or evaluating committee member, later in life e.g., your boss). The report should be at his level: find an appropriate balance between context details and conciseness. In general, the introduction of a technical report should present the context and describe the design objectives (problems to be solved), at a sufficiently high level. \n",
"\n",
"After the problem definition/requirements and an initial analysis that identifies the critical design issues, a typical report would split the problem up into sub-problems (subsystems) which are defined in general terms in the Introduction, and developed individually in the subsequent sections. A picture (block scheme) might be helpful to show the structure and the relations among subsystems. Sometimes, after the introduction a section is needed that describes the background in more technical or mathematical detail, or analyzes the problem in more detail. For the mid-term report, that structure is probably overkill.\n",
"\n",
"In general, an introduction may also contain a literature overview, references to similar work, e.g., how similar problems have been solved and what are the limitations of those solutions (thus motivating your present work), but that is probably not needed for IP3.\n",
"\n",
"The sections on subsystems may follow the same structure but at a more detailed level.\n",
"\n",
"Place your report in context: \"This report describes ---\". You don't need to motivate the project (i.e., don't include a text on the relevance of autonomous driving). ChatGPT-generated text is not appreciated.\n",
"\n",
"**Sections:** You can probably skip to report on Module 1, but place the results of the other Modules each in a separate section. You don't have to repeat everything from a module, but give sufficient context to make your report independently readable. Embed the assignments in a natural way, and give them intelligible names (not \"Task 1\").\n",
"\n",
"For each section, think of the following aspects:\n",
"\n",
"- *Specifications:* What is the objective? What is given already? You may also define notation here.\n",
"- *Analysis:* What are the one or two key problems that drive the design? How can these be addressed?\n",
"- *Design:* What needs to be designed? What is your approach? Make clear what was already given and what is added by you. \n",
"It is highly appreciated if you include an analysis that explains how accurate your system could be (or will be), in view of hardware limitations. This analysis is then backed up by the verification stage.\n",
"- *The resulting design:* This could comment on the implementation, refer to the main variables that you use, etc; generally after reading this, a reader should be able to quickly grasp the Python code in the Appendix.\n",
"\n",
"- **Testing/verification:** How do you test your solution is functional and meets the specifications? What did you measure/observe? Present your results (e.g., measurement results, a Python plot), describe what you see in each plot, and then what you can conclude from this. Don't be naive: things don't work exactly how you design them. Do the debugging systematically. Be sure the plots have correct labels on the axis, and a legend on the line types if you use multiple lines in a single plot. Check the font size; the plot should be readable.\n",
"\n",
"If something doesn't work, what is your hypothesis on the problem? How would you verify that hypothesis?\n",
"\n",
"It is certainly not appreciated if you only say \"It works\" or \"It didn't work\" without documenting this first (i.e., show results/plots/evidence and discuss what is seen).\n",
"\n",
"**For IP3, particular emphasis will be placed on the testing aspect. You cannot make claims unless you provide evidence. Your overall system will fail if a subsystem fails, and the purpose of testing is to rule out possible causes of failure.**\n",
"\n",
"- *Conclusions:* Each section finished with a conclusion summarizing the results of the Module in a few lines, including claims on expected accuracy. This captures what members of the other sub-group need to know when they use your Module as a black box tool.\n",
"\n",
"**System Integration:** This will be added in the final report. Apart from the integration aspects, this is a natural place to describe a GUI and its functionality. Include a screenshot.\n",
"\n",
"Also in this section, you need a subsection on Verification, now on the complete design (overall test). The results of the final challenge may be added here as well. \n",
"\n",
"**Conclusions:** Clearly mention if something doesn't work. This is still the formal part of the report, so keep the language sufficiently formal until this point.\n",
"\n",
"**Discussion:** Here you can include information about the (group) process and any other useful feedback. This part can also include your original planning (work division over team members and time) and the actual outcome. See the Kickoff document: how did that work out? Did everyone contribute?\n",
"\n",
"The material under Discussion can be more informal. In a real (formal) report like a thesis, these sections would be omitted or worked into as an Acknowledgement or Postscript.\n",
"\n",
"**Appendix:** Include Python code, the Jupyter notebooks, and the details on your design that may be helpful.\n",
"- Structure your code into functions. \n",
"- Functions should have a header block that explains what the function does and what the input/output parameters are. Also include author and date information (version, history).\n",
"- Describe data structures and other global variables in sufficient detail.\n",
"\n",
"Your appendix has to be sufficiently complete such that the experiments are reproducible by others."
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

334
8_Module_5.ipynb Normal file
View File

@ -0,0 +1,334 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](pictures/header.png)\n",
"\n",
"[Table of Contents](0_Table_of_Contents.ipynb)\n",
"\n",
"# Chapter 8: Module 5 - Navigating KITT and Final Challenges \n",
"\n",
"**Contents:**\n",
"* [Define The Problem](#define-the-problem)\n",
"* [Obstacle Avoidance](#obstacle-avoidance)\n",
"* [Putting it all together](#putting-it-all-together)\n",
"* [Final Challenge](#final-challenge)\n",
"* [Final report](#final-report)\n",
"* [Final Presentation and discussion](#final-presentation-and-discussion)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"This chapter contains information that you will need when preparing for the second part: the final challenge. The ultimate goal is to let KITT drive from an initial (known) location A to a given target location B. In this final challenge you are on your own and no explicit steps are provided to help you through the process. Nevertheless, a few hints are given in this chapter on how you could attack the problem (but many alternative solution approaches exist).\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Define the problem\n",
"\n",
"> **The problem can be described as:** Write a Python program for KITT that enables autonomous driving from a given starting point A to a designated target point B. The car state in point A is known.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" As target accuracy, aim to arrive at B within a distance of 30 cm (measured from the center of the beacon). For the moment, you can assume there are no obstacles in the field, and that point B is reachable without driving off the field or needing to drive backwards. Once you have mastered to drive from A to B, extend your solution to drive to a third point C as well. This is essentially the same, except that the initial state (at B) is only approximately known, and depending on how you arrived at B, you might not be able to make the turn to C within the field boundaries."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### What information is important?\n",
"\n",
"1. At the starting point, you know both the car's position and its orientation. This is your initial condition. At location A, this information is accurate, but after driving for some time, it is approximate.\n",
"2. The target location is known, as it is provided. However, the car's orientation upon reaching the target is arbitrary.\n",
"3. You have created an algorithm to estimate the car's location, providing an estimate based on the moment the recording was made. However, keep in mind that the car may have moved while the recordings are being processed (unless you stop the car while localizing it). You also do not know the car's orientation. How can it be estimated? (Taking the difference of two positions is not accurate if you are making a turn, or if the positions are close to each other and both inaccurate.)\n",
"4. You have developed a virtual car model. You can use it in three ways: \n",
" - Track your current position and orientation (give the model the same commands as you give to your real car); \n",
" - Determine which commands should be given to reach the destination.\n",
" - Debug your algorithms: instead of testing on the real car, first test using the model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### What do you need to make ? \n",
"\n",
"1. **Path planning:** You need to determine a path from point A to point B, which would ideally be a straight line. However, keep in mind that KITT has a specific orientation, and you can’t move or turn it exactly as you wish. The path to B is typically a circle followed by a straight line. You could choose to use a circle with a small diameter, and stop turning when the car is headed into the right direction. Alternatively, you could calculate a wide circle such that no straight driving is needed. \n",
"In both cases, you must calculate how long you want to drive on the intended circle (which depends on your selected velocity as well).\n",
"\n",
"2. **Path Following:** After defining a path, you need to give commands for the car to stay on the path. The way you do this will vary based on your strategy. E.g., at any new position, you could recalculate your optimal path (drive on a small circle, etc), or you can estimate an error and do error feedback. You will see examples of this in a later course (EE2S2 Systems and Control). A simple technique is to define the error as the difference in the current orientation angle and the angle of the direct line towards the target, and apply proportional error feedback: a larger error gives a larger steering action. You will have to worry about stability.\n",
"\n",
"3. **Location estimation** As you may have already noticed, having a path and a controller alone can make it seem like the TDOA-based localization function is unnecessary. However, relying solely on a path and a path-following mechanism (i.e., your virtual model) is essentially like driving blind, knowing only the start and end points (an open-loop controller). The model will allow you to drive for about 2 seconds and predict where you are, after that its accuracy is probably insufficient. Therefore, you will have to regularly estimate your position using the TDOA algorithm. The optimal blend of your TDOA-estimated position with the model-based position estimate requires more advanced control theory (Kalman filters) - this is a topic for the MSc. Some of this is presented in the [state tracking notebook](./state-tracking.ipynb). \n",
"\n",
"As a simple approach, you could update the state in your virtual model by the location estimates from the Location module once they become available. If you drive while estimating locations, then you should realize that that module will give your location from some time ago; a proper update takes that into account. \n",
"\n",
"For some guidance on how to plan your path or control the car, you could refer to the notebooks on [path planning](Path-planning.ipynb) and [control](Controller.ipynb). However, this material is readable only after you have done the course EE2S2 Systems and Control."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Note:**\n",
"> Keep in mind that path planning and path following don't necessarily have to be completely separated parts. \n",
"> The control system and path planning can be merged in simpler designs.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Defining your approach\n",
"\n",
"At this time it merits to brainstorm with your entire group on the approach\n",
"that you will take. For your selected approach, determine whether it will do the job (and under what assumptions/conditions). \n",
"\n",
"You can implement various strategies. The simplest is to move step by step: let the vehicle drive for 1 or 2 seconds, stop, estimate the location using the beacon, determine a control (= steering) action, and then continue moving. You won't really need your virtual model for this. At the other extreme, you could base yourself entirely on your car model. \n",
"\n",
"If you opt for localization while driving (obviously faster, much more realistic and comfortable for passengers), then you need a main loop that alternates localization with control. It would be nicer to do control continuously, while the localization function is regularly called e.g. using interrupts. In that case, the audio read function should probably be _non-blocking_, so that control can continue while the audio samples are collected. This will require next-level Python programming!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Note 1:**\n",
"> You can opt for a simple and robust \"decision tree\" approach (i.e., based on many if-then-else statements), that will be able to handle the expected situations but not more than that, or a more thoughtful approach based on control theory, that is much more robust but also more difficult to implement. Note that decision trees are very hard to debug: for each if-then-else statement, you need to test both branches. \n",
"\n",
"> **Note 2:**\n",
"> This high-level system design is part of system engineering, i.e., with the entire system in mind, define an approach at a high level that will do the job, and define specifications of each constituting subsystem. These parts can then be designed by specialists that don’t have to know or consider the entire system. If the parts are tested and verified to meet their specifications, then the entire system is supposed to work. \n",
"\n",
"> For the current system, the hardest to specify are the timing conditions. E.g., if you specify a function that returns your state (position, orientation, velocity) from two past location estimates, then keep in mind that simple approaches will not give you the orientation and velocity at the current time, but at some time ago. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"\n",
"**Deliverable**\n",
"\n",
"In your final report, document your selected approach, and the alternatives that you considered, plus your\n",
"motivation for the selection.\n",
"\n",
"Document the foreseen consequences of this choice. Draw a block scheme that shows what software\n",
"blocks should interact, and define the interaction (e.g., variables).\n",
"\n",
"Your solution will probably contain a finite state machine and/or a loop. An important consideration is\n",
"the timing of this loop. Document your analysis of this in sufficient detail. How often do you intend to\n",
"measure your location, keeping in mind the constraints and trade-offs? Consider also the various\n",
"delays in the system. If you obtain a location fix, how old is that information? If you estimate velocity\n",
"from two locations, then to what time point does that velocity refer? How fast should one iteration of the\n",
"loop be? (If it is too fast, then you won’t have new location information, but if it is too slow, then you\n",
"might miss your target.) Can you merge location fixes from the audio beacon with predictions of your\n",
"position using your car model?\n",
"\n",
"The text on this in your final report can be placed into an initial section “Problem definition and analysis”, or “Problem analysis and high-level design”, depending on how you want to organize your report. Alternatively, it can be placed after the localization and car modeling sections, if you need to use information from those sections.\n",
"\n",
"In the final report, document your path planning solution. Illustrate this with examples of generated routes under varying conditions. You can use your car model to debug your algorithms!\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Obstacle avoidance\n",
"\n",
"After you are able to confidently drive from A to B to C (etc), you can try the next challenge: obstacle avoidance. In its generality, this is an advanced topic.\n",
"\n",
"You have the parking sensors to help you detect obstacles, and also the perimeter of the field can be considered an (invisible) obstacle. Once you detect an obstacle, you need to steer around it. You can have the following options:\n",
"\n",
"- The most structured approach is to create a map of the environment. For example, you can store a 2D grid as an array, which serves as occupancy map. In this grid, any locations where obstacles are detected are marked with a value of 1, indicating that no path can be planned through those areas. When an obstacle is encountered, you halt and generate a new path from your current position to the target, factoring in the updated map.\n",
" \n",
" 📝 **Note:** Making a map is probably most complete approach, but it is difficult and time-consuming.\n",
"\n",
"- Instead, a simple approach is to define a new planned trajectory around the obstacle (possibly requiring driving backwards and then making a wide circle). Typically, students follow a *decision tree approach*: lots of if-then-else statements. When driving backwards, you risk driving off the field.\n",
"\n",
" 📝 **Note:** A decision tree works in simple cases, but the disadvantage is that it is hard to debug, and the solutions are often not general.\n",
"\n",
"- Study control literature on obstacle avoidance that use *artificial potential fields*. This is a general approach that essentially defines a penalty function around obstacles and then finds optimal trajectories that minimize the \"cost\".\n",
"\n",
"📝 **Note:** You can find some advice on this in the [notebook on obstacle avoidance](./obstacle.ipynb). More complex approaches rely on control theory, which will be covered in later courses.\n",
"\n",
"Here is a summary of these suggestions: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Obstacle avoidance options\n",
"\n",
"| **Approach** | **Advantage** | **Disadvantage** |\n",
"|-------------------------------------|-----------------------------------------------------|-----------------------------------------------------|\n",
"| **Create a map of the environment** | Most complete and accurate approach. | Difficult and time-consuming to implement. |\n",
"| **Decision tree approach** | Simple and works well in basic situations. | Hard to debug and not generalizable for complex cases. |\n",
"| **Artificial potential fields or other control algorithms** | General approach that can handle complex scenarios. | More advanced and requires control theory knowledge. |\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Putting it all together \n",
"\n",
"Before you can start with the integration part, it is necessary to\n",
"have completed all the previous subsystems. \n",
"To link these parts together, you must communicate with the complete group about the responsibilities. The assignment during system integration is to complete the big final design such that you can complete the challenges. Read this chapter completely to know what they are.\n",
"\n",
"Focus on the basic challenges first. Once these have been implemented, you can work on the obstacle avoidance and anti-collision system. \n",
"\n",
"Take it step-by-step, and keep it simple and structured. You'll quickly reach a level where the car shows unexpected behavior and nobody understands why. You can only manage that if each subgroup delivers fully tested functions and subsystems. \n",
"\n",
"For the control system, you can choose to plan a route but you are not required to do so. Important to note is that the location estimates from the locating algorithm must be taken into account during the challenge as the integration of these two is the central focus of the system. \n",
"\n",
"When integrating the control system with the locating algorithm, it is important keep in mind the time it takes to run your control loop after each locating attempt and consider how you can ensure that you always record a full beacon waveform. Ideally, you drive your car while estimating locations. If necessary, it is permitted to stop the car while measuring. This is much easier to program, but you certainly won't be competing for speed that way.\n",
"\n",
"After you have completed the integration of the control and locating algorithm, you can attempt an implementation of the obstacle detection and avoidance system for challenges C and D. \n",
"\n",
"If you have time, we recommend to design a GUI; see the figure below for an example. This will facilitate to enter parameters for each challenge, and help you to keep an overview on what's going on. But if you have a tight control loop, you'll have to watch out that updating the graphics is not taking too much time.\n",
"\n",
"Since you have a virtual car model, you could also use a switch and have the virtual car drive the track, so that you can test the control and obstacle detection performance on those moments that the real field is not available.\n",
"\n",
"Do not postpone to document your work. The final report is needed very shortly after the demonstrations.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> 📘 **Suggested workflow:**\n",
"> \n",
"> 1. **Complete all subsystems** before starting integration.\n",
"> 2. **Communicate** with the group to assign responsibilities.\n",
"> 3. Focus on **basic challenges** first.\n",
"> 4. **Integrate the control system** with the locating algorithm, ensuring real-time performance.\n",
"> 5. Consider **stopping the car** during measurements for simplicity.\n",
"> 6. Once basic integration is done, implement **obstacle detection** for advanced challenges.\n",
"> 7. If time permits, create a **GUI** for parameter management.\n",
"> 8. Test the system using the **virtual car model** when real testing isn't possible.\n",
"> 9. **Document** your work as you progress."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Final challenge\n",
"\n",
"The final challenge for IP3 consists of four challenges and a fifth \"free challenge\". In order to get a passing grade, you have to successfully pass at least the first challenge. With every other challenge you pass you will get a higher grade. You will have maximum two attempts for each challenge. If you fail a challenge, you may not compete in further challenges, except for the \"free challenge\".\n",
"\n",
"The car reference position is the beacon location.\n",
"Before each challenge you are allowed to measure the field and the position of the destination (and waypoints). A challenge starts at a given (known) starting position at the edge of the field. The orientation of KITT is always 90° with respect to the edge of the field. Once the start command has been issued you may not touch KITT nor the PC except for an emergency stop.\n",
"\n",
"KITT may stop everywhere and as many times as needed. There is no time limit except for the fact that everything, including preparation and cleaning up, should happen within $30$ minutes. Needless to say, you must use the audio beacon for locating KITT, and the ultrasonic sensors for obstacle detection (no \"open loop\" solutions allowed).\n",
"\n",
"**Challenge A** (65 points)\n",
"- KITT drives from the starting position to a specific point A in the field.\n",
" \n",
"- Once KITT reaches the destination it must stop and the PC must give a signal.\n",
"\n",
"**Challenge B** (10 points)\n",
"- KITT drives from the starting position to point A via another point B in the field.\n",
"- When KITT reaches the waypoint (B) the examiner must have the time to measure the distance; you should let the car stop for 10 seconds. \n",
"- When KITT reaches the destination (A) it must stop and the PC must give a signal.\n",
"\n",
"**Challenge C** (10 points)\n",
"- KITT drives from the starting position to a specific point A in the field.\n",
"- When KITT reaches the destination (A) it must stop and the PC must give a signal.\n",
"- There is an obstacle on route (two paper bins on top of each other) that has to be avoided.\n",
"- When KITT sees the obstacle, its position must be remembered.\n",
"\n",
"**Challenge D** (8 points)\n",
"- KITT drives from the starting position to a specific point A in the field.\n",
"- When KITT reaches the destination it must stop and the PC must give the signal.\n",
"- There is another car involved (stationary but with working beacon), this car has to be avoided. \n",
"\n",
"**Free challenge** (7 points)\n",
"- Invent your own challenge and impress us! For example, drive form A to B to C where after B you will need to drive backwards to make the turn.\n",
"- Points awarded depending on the difficulty and creativity of the task.\n",
"\n",
"**Grading**\n",
"\n",
"- If you complete each task perfectly you receive the total amount of points. Penalty points are deducted for missing the target or hitting the obstacle. \n",
"\n",
"ℹ️ <font color='orange'>The grading for a task cannot be negative. </font>\n",
"\n",
"- The measurements are done with respect to the center of the louspeaker on top of KITT. \n",
"\n",
"- In *Challenge A* you may miss the destination by $30$ cm but for every additional $10$ cm you will lose $5$ points. You can't lose more than $15$ points.\n",
"\n",
"- In *Challenges B-D* you may miss the targets by $30$ cm and for every additional $10$ cm you will lose $2.5$ points, this holds for both the way point and the destination.\n",
"\n",
"- For each time KITT hits an obstacle or another car you will lose $2.5$ points.\n",
"\n",
"**Bonus:** The time you take to complete the challenges will be measured. The fastest team will receive 10 bonus points and the second fastest will receive 5 bonus points. \n",
"\n",
"ℹ️ <font color='orange'>It is not possible to receive more than 100 points for the whole competition.</font>\n",
"\n",
"The figure below is a example depiction of the Challenges (*On the old television series, KARR is the archenemy of KITT*)\n",
"\n",
"![alt text](pictures/FinalChallenge.png)\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Final report\n",
"\n",
" Instructions for the final report are similar to those of the midterm report (see Mid-term report chapter). Aim for a **well-structured**, compact yet complete report of about 30 pages (plus Python code in an appendix). Do not forget to systematically report on testing/verification: how do you test (each subsystem, and the entire system), what are the results from the test, what do you conclude. Include the results of the final challenge in the report as well. Note that the final challenge is not a test, rather it is a demonstration. With extensive testing, these results won't be a surprise to the reader!\n",
" \n",
"The report is judged by committee members that are not indepth familiar with IP3, or the manual. Your report has to be sufficiently self-contained.\n",
" \n",
"The submission deadline is listed on Brightspace, typically it is one day after the final challenge. Submit your report using the corresponding submission folder.\n",
"\n",
"## Final presentation and discussion\n",
"\n",
" In week 10 (consult Brightspace for the exact date), you present and defend your final report in front of an examination committee. The examinators will ask questions about your design choices and aspects of teamwork. This will be part of your grade.\n",
"\n",
"The presentation lasts at most 10 min. Focus on the highlights and special features of the design, and mention the work breakdown and distribution of tasks to team members.\n",
"\n",
"The examination will last about 30 min. After the examination you will be asked to fill in a peer review form. Individual grades are differentiated depending on staff observations and the outcome of the peer review.\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

BIN
KITT_Simulator/GUI/car.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.2 MiB

BIN
KITT_Simulator/GUI/explosion.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 MiB

115
KITT_Simulator/dynamics_simulator.py Normal file
View File

@ -0,0 +1,115 @@
import time
import numpy as np
# try:
# from KITT_Simulator.shared_state import SharedState
# except ImportError:
# from shared_state import SharedState
# else:
# raise ImportError("Could not import the required modules.")
class Dynamics:
def __init__(self, start_state):
"""Initializes the Dynamics class with the start state and parameters."""
self.start_time = time.time()
self.state = start_state
self.mass = 5.6 # Mass of the car in kg
self.L = 1 # Length of the car in cm
self.T = 1 # Track width in cm
self.F = {}
self.b = 5
self.R = {}
try:
with open("simulator_data/motor_parameters.txt", "r") as f:
for line in f:
line = line.split(sep=",")
self.F[int(line[0])] = float(line[1])
with open("simulator_data/servo_parameters.txt", "r") as f:
for line in f:
line = line.split(sep=",")
self.R[int(line[0])] = float(line[1])
except(FileNotFoundError):
with open("KITT_Simulator/simulator_data/motor_parameters.txt", "r") as f:
for line in f:
line = line.split(sep=",")
self.F[int(line[0])] = float(line[1])
with open("KITT_Simulator/simulator_data/servo_parameters.txt", "r") as f:
for line in f:
line = line.split(sep=",")
self.R[int(line[0])] = float(line[1])
else:
raise FileNotFoundError("Could not find the motor and servo parameters.")
self.motor_command = 150
self.servo_command = 150
self.update_state()
def update_state(self):
"""Updates the state of the car based on the motor and servo commands."""
# Get the parameters for the motor and servo commands
F, b, R, direction = self.map_command(self.motor_command, self.servo_command) # F = motor force, b = drag coefficient, R = turn radius, direction = left or right turn
# Calculate time since last update
update_time = time.time()
delta_t = update_time - self.state.last_update
self.state.last_update = update_time
# Calculate how far the car moves during delta_t
arc_length = self.calculate_arc_length(F, b, delta_t)
self.arc_length = arc_length
self.update_position(arc_length, R, direction)
self.motor_command = self.state.motor_command
self.servo_command = self.state.servo_command
def map_command(self, motor_command, servo_command):
"""Converts a command into motor force and turn radius."""
try:
motor_force = self.F.get(motor_command)
turn_radius = self.R.get(servo_command)
except:
motor_force = 0
turn_radius = 0
direction = "Left" if float(servo_command) > 150 else "Right"
return motor_force, self.b, turn_radius, direction
def calculate_arc_length(self, F, b, delta_t):
"""Calculates the arc length the car moves during delta_t."""
arc_length = (F)/(b)*delta_t + (F*self.mass-self.mass*b*self.state.v)/(b**2)*np.exp(-(b)/(self.mass)*delta_t) - (F*self.mass-self.mass*b*self.state.v)/(b**2)
# Calculate the new velocity of the car
self.state.v = (F)/(b) + (self.state.v - (F)/(b))*(np.exp(-(b)/(self.mass)*delta_t))
return arc_length
def update_position(self, arc_length, R, direction):
"""Updates the position of the car based on the arc length and turn radius."""
x0 = self.state.x
y0 = self.state.y
theta0 = self.state.theta
if R != 0:
arc_angle = arc_length / (R) # angle between start and end of path along the circle
delta_x = R - R*np.cos(arc_angle)
delta_y = R*np.sin(arc_angle)
if direction == 'Left':
delta_x = -delta_x
arc_angle = -arc_angle
x = x0 + delta_x*np.cos(theta0 - np.pi/2) + delta_y*np.cos(theta0)
y = y0 + delta_x*np.sin(theta0 - np.pi/2) + delta_y*np.sin(theta0)
theta = theta0 - arc_angle
else:
x = x0 + arc_length*np.cos(theta0)
y = y0 + arc_length*np.sin(theta0)
theta = theta0
if theta < -np.pi:
theta += 2*np.pi
if theta > np.pi:
theta -= 2*np.pi
# Update the state
self.state.x = x
self.state.y = y
self.state.theta = theta

237
KITT_Simulator/gui.py Normal file
View File

@ -0,0 +1,237 @@
import math
import time
import numpy as np
from ipycanvas import Canvas, hold_canvas
from shapely.geometry import Point, Polygon
from ipywidgets import Image
from IPython.display import display
class GUI:
def __init__(self, state):
self.state = state # Store the car state
# Define the canvas size with padding (20 pixels on each side)
self.padding = 20
self.field_size = 480
self.canvas_size = 2 * self.padding + self.field_size
self.mic_positions = np.array([[0, 0], [0, self.field_size], [self.field_size, self.field_size], [self.field_size, 0], [0, self.field_size / 2]])
# Create the main canvas where everything will be drawn
self.canvas = Canvas(width=self.canvas_size, height=self.canvas_size)
# Draw the field area
self.canvas.fill_style = '#DDDDDD' # Background color for the field
self.canvas.fill_rect(self.padding, self.padding, self.field_size, self.field_size) # Draw the 480x480 field
# Car dimensions
self.car_width = 60
self.car_height = 30
# Create a pre-rendered canvas for the car (to optimize drawing speed)
self.car_canvas = Canvas(width=self.car_width, height=self.car_height)
try:
self.car_canvas.draw_image(Image.from_file('GUI/car.png'), 0, 0, self.car_width, self.car_height)
except FileNotFoundError:
self.car_canvas.draw_image(Image.from_file('KITT_Simulator/GUI/car.png'), 0, 0, self.car_width, self.car_height)
# Create a pre-rendered canvas for the explosion
self.explosion_canvas = Canvas(width=200, height=200)
try:
self.explosion_canvas.draw_image(Image.from_file('GUI/explosion.png'), 0, 0, 200, 200)
except FileNotFoundError:
self.explosion_canvas.draw_image(Image.from_file('KITT_Simulator/GUI/explosion.png'), 0, 0, 200, 200)
self.simulation_running = True
self.prev_positions = []
def update(self):
if not self.simulation_running:
return
collision_detected = self.check_collision()
self.calculate_distance_to_wall()
with hold_canvas(self.canvas):
self.canvas.clear() # Clear the main canvas to redraw the car
# Redraw the field
self.canvas.fill_style = '#DDDDDD'
self.canvas.fill_rect(self.padding, self.padding, self.field_size, self.field_size)
# Draw the microphones
self.canvas.fill_style = 'black'
for mic_x, mic_y in self.mic_positions:
mic_x = mic_x + self.padding
mic_y = self.field_size - mic_y + self.padding
self.canvas.fill_circle(mic_x, mic_y, 3)
self.canvas.fill_style = 'black'
self.canvas.fill_text(f"M{self.state.motor_command}", self.field_size - 20, 20 + self.padding)
self.canvas.fill_text(f"D{self.state.servo_command}", self.field_size - 20, 30 + self.padding)
self.canvas.fill_text(f"L{self.state.dist_L:.2f}", self.field_size - 20, 40 + self.padding)
self.canvas.fill_text(f"R{self.state.dist_R:.2f}", self.field_size - 20, 50 + self.padding)
# Calculate the car position with the bottom-left origin and center the car
car_x = self.state.x + self.padding
car_y = self.field_size - self.state.y + self.padding # Flip y-axis for bottom-left origin
# Track the previous positions
self.prev_positions.append((car_x, car_y))
# Draw the car's previous 1000 positions
for i in range(len(self.prev_positions)):
if i == len(self.prev_positions) - 1:
break
self.canvas.fill_style = 'grey'
self.canvas.fill_circle(self.prev_positions[i][0], self.prev_positions[i][1], 1)
if collision_detected:
# Draw the explosion image at the car's location
self.canvas.draw_image(self.explosion_canvas, car_x - 100, car_y - 100, 200, 200)
self.simulation_running = False # Stop simulation after collision
else:
# Apply rotation and draw the pre-rendered car canvas
self.canvas.save() # Save the current canvas state
self.canvas.translate(car_x, car_y) # Translate to the car's position
self.canvas.rotate(-self.state.theta + np.pi) # Rotate around the car's center
# Draw the pre-rendered car canvas (optimized for speed)
self.canvas.draw_image(self.car_canvas, -self.car_width / 2, -self.car_height / 2)
if self.state.beacon:
# Draw a blue circle on the car
self.canvas.fill_style = 'blue'
self.canvas.fill_circle(0, 0, 5)
self.canvas.restore() # Restore the canvas state to prevent affecting other drawings
def check_collision(self):
# Define the car polygon based on the current state
car_polygon = Polygon([
(self.state.x - self.car_width / 2, self.state.y - self.car_height / 2),
(self.state.x + self.car_width / 2, self.state.y - self.car_height / 2),
(self.state.x + self.car_width / 2, self.state.y + self.car_height / 2),
(self.state.x - self.car_width / 2, self.state.y + self.car_height / 2)
])
# Define the field boundaries as a polygon
field_polygon = Polygon([
(self.padding, self.padding),
(self.padding + self.field_size, self.padding),
(self.padding + self.field_size, self.padding + self.field_size),
(self.padding, self.padding + self.field_size)
])
# Check if the car is outside the field boundaries
if not field_polygon.contains(Point(self.state.x, self.state.y)):
print("Car is outside the field boundaries!")
self.simulation_running = False
return True
return False
def calculate_distance_to_wall(self):
"""
Calculates the distance from the front-left and front-right corners of the car to the nearest wall
the car is facing, considering the car's orientation within a square field.
"""
# Car's dimensions
w = self.car_width
h = self.car_height
# Car's position and orientation
x_car = self.state.x
y_car = self.state.y
theta = self.state.theta
cos_theta = np.cos(theta)
sin_theta = np.sin(theta)
# Front-left corner in local coordinates (corrected to front)
x_fl_local = w / 2
y_fl_local = h / 2
# Front-right corner in local coordinates (corrected to front)
x_fr_local = w / 2
y_fr_local = -h / 2
# Compute global positions of the front-left corner
x_fl_global = x_car + x_fl_local * cos_theta - y_fl_local * sin_theta
y_fl_global = y_car + x_fl_local * sin_theta + y_fl_local * cos_theta
# Compute global positions of the front-right corner
x_fr_global = x_car + x_fr_local * cos_theta - y_fr_local * sin_theta
y_fr_global = y_car + x_fr_local * sin_theta + y_fr_local * cos_theta
# Direction vector (normalized)
dir_x = cos_theta
dir_y = sin_theta
# Function to compute distance to walls from a point
def distance_to_walls(x0, y0, dir_x, dir_y):
t_values = []
# Left wall x = 0
if dir_x != 0:
t = (0 - x0) / dir_x
if t > 0:
y = y0 + t * dir_y
if 0 <= y <= self.field_size:
t_values.append(t)
# Right wall x = self.field_size
if dir_x != 0:
t = (self.field_size - x0) / dir_x
if t > 0:
y = y0 + t * dir_y
if 0 <= y <= self.field_size:
t_values.append(t)
# Bottom wall y = 0
if dir_y != 0:
t = (0 - y0) / dir_y
if t > 0:
x = x0 + t * dir_x
if 0 <= x <= self.field_size:
t_values.append(t)
# Top wall y = self.field_size
if dir_y != 0:
t = (self.field_size - y0) / dir_y
if t > 0:
x = x0 + t * dir_x
if 0 <= x <= self.field_size:
t_values.append(t)
if t_values:
return min(t_values)
else:
return float('inf') # No intersection in the positive t direction
# Compute distances from the front-left and front-right corners
dist_L = distance_to_walls(x_fl_global, y_fl_global, dir_x, dir_y)
dist_R = distance_to_walls(x_fr_global, y_fr_global, dir_x, dir_y)
# Update the state with the computed distances
self.state.dist_L = dist_L
self.state.dist_R = dist_R
def display(self):
display(self.canvas) # Ensure the canvas is displayed in the notebook
def stop(self):
self.simulation_running = False
self.__del__()
def __del__(self):
pass
# Usage:
# state = State()
# gui = GUI(state)
# gui.display()
# for i in range(100):
# state.x = i
# state.y = 2 * i
# state.theta = np.pi / 2 + i / 100
# gui.update()
# time.sleep(0.5)
# gui.stop()

75
KITT_Simulator/gui_tester.ipynb Normal file
View File

@ -0,0 +1,75 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from gui import GUI\n",
"from shared_state import SharedState\n",
"\n",
"import time\n",
"import numpy as np"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "b37d900d82e94817a23ed79f23068d41",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Canvas(height=520, width=520)"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"state = SharedState()\n",
"gui = GUI(state)\n",
"gui.display()\n",
"\n",
"for i in range(100):\n",
" state.x = i + 20\n",
" state.y = 2*i + 20\n",
" state.theta = np.pi/2 - i/100\n",
" gui.update()\n",
" time.sleep(0.1)\n",
" if i == 50:\n",
" state.beacon = True\n",
"\n",
"gui.stop()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "OpenEdVenv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

139
KITT_Simulator/serial_simulator.py Normal file
View File

@ -0,0 +1,139 @@
import time
import threading
import numpy as np
import matplotlib.pyplot as plt
import logging
import queue # Import queue to handle inter-thread communication
from KITT_Simulator.shared_state import SharedState
from KITT_Simulator.gui import GUI
from KITT_Simulator.dynamics_simulator import Dynamics
# try:
# from KITT_Simulator.shared_state import SharedState
# from KITT_Simulator.gui import GUI
# from KITT_Simulator.dynamics_simulator import Dynamics
# except ImportError:
# from shared_state import SharedState
# from GUI_notebook import GUI
# from dynamics_simulator import Dynamics
# else:
# raise ImportError("Could not import the required modules.")
class Serial:
"""Class to simulate serial communication with the car."""
def __init__(self, port, baudrate, rtscts=True, x=240, y=30, theta=np.pi/2, update_freq=20):
"""Initializes the Serial class with port, baudrate, and initial state, and starts the dynamics thread."""
self.port = port
self.baudrate = baudrate
self.rtscts = rtscts
self.update_freq = update_freq
self.send_buffer = [] # Buffer for storing commands to be sent
self.state = SharedState(x, y, theta)
self.dynamics = Dynamics(self.state)
self.gui = GUI(self.state) # Initialize the GUI
self.gui.display()
self.update_thread = threading.Thread(target=self.run_dynamics)
self.update_thread.daemon = True
self.stop_thread = threading.Event()
self.update_thread.start()
def write(self, command):
"""Writes a command to the serial port and updates the shared state accordingly."""
with self.state._lock:
logging.debug(f"Received command: {command}")
match command[0:1]:
case b'M':
motor_command = int(command[1:-1])
self.state.motor_command = motor_command
case b'D':
servo_command = int(command[1:-1])
self.state.servo_command = servo_command
case b'A':
if int(command[1:-1]) == 1:
self.state.beacon = True
elif int(command[1:-1]) == 0:
self.state.beacon = False
case b'S':
left_distance = self.state.dist_L
right_distance = self.state.dist_R
voltage = 11.5
audio_code = 0xABCDEF00
carrier_frequency = 5678
bit_frequency = 1234
repetition_count = 1337
match command[1:2]:
case b'd':
status = "USL{}\nUSR{}\n\x04".format(left_distance, right_distance)
case b'v':
status = """VBATT{:.2f}V\n\x04""".format(voltage)
case _:
status = '**************************\n'
status += '* Audio Beacon: {}\n'.format('on' if self.state.beacon else 'off')
status += '* c: {:#x}\n'.format(audio_code)
status += '* f_c: {}\n'.format(carrier_frequency)
status += '* f_b: {}\n'.format(bit_frequency)
status += '* c_r: {}\n'.format(repetition_count)
status += '**************************\n'
status += '* PWM:\n'
status += '* Dir. {}\n'.format(self.state.servo_command)
status += '* Mot. {}\n'.format(self.state.motor_command)
status += '**************************\n'
status += '* Sensors:\n'
status += '* Dist. L {} R {}\n'.format(left_distance, right_distance)
status += '* V_batt {} V\n'.format(voltage)
status += '**************************\n\x04'
self.send_buffer.append(status)
case b'V':
version = '*******************************\n' \
'* EPO-4 *\n' \
'*******************************\n' \
'* KITT Simulator Rev. 0.1 *\n' \
'* Audio Firmware Rev. 0.0 *\n' \
'*******************************\n' \
'* Author: M. Rom B.Sc. *\n' \
'* Date: Sep 10, 2024 *\n' \
'*******************************\n\x04'
self.send_buffer.append(version)
def read_until(self, end):
"""Reads from the send buffer until a specified end character is found."""
return self.send_buffer.pop(0).encode() # Return the first element in the send buffer
def run_dynamics(self):
"""Continuously updates the state using dynamics every 50ms."""
update_freq = self.update_freq # Desired updates per second
update_interval = 1 / update_freq # Time per update in seconds
while not self.stop_thread.is_set():
start_time = time.time() # Start time of this loop
self.dynamics.update_state()
self.gui.update()
elapsed_time = time.time() - start_time # Time taken for this update loop
sleep_time = update_interval - elapsed_time
if sleep_time > 0:
time.sleep(sleep_time) # Sleep only if we have time remaining in the frame
else:
print("WARNING: Dynamics thread running too slow. Consider reducing the update frequency.")
def close(self):
"""Closes the serial connection."""
if self.update_thread.is_alive():
self.stop_thread.set()
self.update_thread.join()
SharedState.reset()
def __del__(self):
"""Destructor to clean up when the Serial object is deleted."""
self.close()

37
KITT_Simulator/shared_state.py Normal file
View File

@ -0,0 +1,37 @@
from dataclasses import dataclass
import threading
import numpy as np
import time
@dataclass
class State:
"""Class to represent the current state of the car."""
x: float # x position in cm
y: float # y position in cm
theta: float # angle in radians
v: float = 0 # velocity in cm/s
last_update: float = time.time() # time since last position compute in ms
motor_command: int = 150 # last received motor command
servo_command: int = 150 # last received steering command
beacon: bool = False # beacon status on/off
dist_L: float = 0 # left distance sensor reading in cm
dist_R: float = 0 # right distance sensor reading in cm
_lock = threading.Lock() # lock to ensure thread safety
class SharedState:
"""Singleton class to manage shared state with thread safety."""
_instance = None
def __new__(cls, x=240, y=30, theta=np.pi/2):
"""Creates a new instance of State if none exists, otherwise returns the existing one."""
if cls._instance is None:
cls._instance = State(x, y, theta)
return cls._instance
@classmethod
def reset(cls):
"""Resets the singleton instance."""
cls._instance = None
def __del__(cls):
cls._instance = None

31
KITT_Simulator/simulator_data/motor_parameters.txt Normal file
View File

@ -0,0 +1,31 @@
135, -747.9707140722312
136, -697.1135310316458
137, -626.1056068618782
138, -539.6300413380377
139, -442.3699342366308
140, -339.0083853341639
141, -234.22849440574646
142, -132.71336122835055
143, -39.14608557661995
144, 0
145, 0
146, 0
147, 0
148, 0
149, 0
150, 0
151, 0
152, 0
153, 19.408136868849397
154, 71.51385027728975
155, 139.80808660713956
156, 220.32698805537075
157, 309.1066968208179
158, 402.1833551018499
159, 495.5931050940417
160, 585.3720889985561
161, 667.5564490100369
162, 738.1823273268528
163, 793.2858661483042
164, 828.9032076704316
165, 841.0704940906726

121
KITT_Simulator/simulator_data/pulses_recording.txt Normal file
View File

File diff suppressed because one or more lines are too long

101
KITT_Simulator/simulator_data/servo_parameters.txt Normal file
View File

@ -0,0 +1,101 @@
100, 99.99999999999636
101, 96.24208304010881
102, 93.40605207599856
103, 91.49190710766743
104, 90.4996481351127
105, 90.42927515833526
106, 91.28078817733604
107, 93.05418719211502
108, 95.7494722026704
109, 99.366643209004
110, 103.9057002111158
111, 109.366643209004
112, 115.7494722026704
113, 123.05418719211502
114, 131.28078817733694
115, 140.42927515833617
116, 150.4996481351127
117, 161.49190710766743
118, 173.40605207599947
119, 186.2420830401088
120, 199.99999999999727
121, 214.67980295566213
122, 230.28149190710428
123, 246.80506685432556
124, 264.25052779732323
125, 282.6178747360982
126, 301.9071076706514
127, 322.1182266009828
128, 343.25123152709057
129, 365.30612244897657
130, 388.2828993666408
131, 412.1815622800814
132, 437.0021111893002
133, 462.7445460942963
134, 489.40886699507064
135, 516.9950738916223
136, 545.5031667839512
137, 574.9331456720593
138, 605.2850105559437
139, 636.5587614356064
140, 668.7543983110463
141, 701.8719211822627
142, 735.9113300492581
143, 770.87262491203
144, 806.7558057705801
145, 843.5608726249102
146, 881.2878254750149
147, 919.9366643208978
148, 959.507389162558
149, 999.9999999999964
150, 0
151, 999.9999999999964
152, 959.507389162558
153, 919.9366643208978
154, 881.2878254750149
155, 843.5608726249102
156, 806.7558057705801
157, 770.87262491203
158, 735.9113300492581
159, 701.8719211822627
160, 668.7543983110463
161, 636.5587614356064
162, 605.2850105559437
163, 574.9331456720593
164, 545.5031667839512
165, 516.9950738916223
166, 489.40886699507064
167, 462.7445460942963
168, 437.0021111893002
169, 412.1815622800814
170, 388.2828993666408
171, 365.30612244897657
172, 343.25123152709057
173, 322.1182266009828
174, 301.9071076706514
175, 282.6178747360982
176, 264.25052779732323
177, 246.80506685432556
178, 230.28149190710428
179, 214.67980295566213
180, 199.99999999999727
181, 186.2420830401088
182, 173.40605207599947
183, 161.49190710766743
184, 150.4996481351127
185, 140.42927515833617
186, 131.28078817733694
187, 123.05418719211502
188, 115.7494722026704
189, 109.366643209004
190, 103.9057002111158
191, 99.366643209004
192, 95.7494722026704
193, 93.05418719211502
194, 91.28078817733604
195, 90.42927515833526
196, 90.4996481351127
197, 91.49190710766743
198, 93.40605207599856
199, 96.24208304010881
200, 99.99999999999636

1
KITT_Simulator/simulator_data/silence.txt Normal file
View File

File diff suppressed because one or more lines are too long

149
KITT_Simulator/sounddevice_simulator.py Normal file
View File

@ -0,0 +1,149 @@
import time as time
import numpy as np
import matplotlib.pyplot as plt
try:
from KITT_Simulator.shared_state import SharedState
except:
from shared_state import SharedState
# try:
# from KITT_Simulator.KITT_Control.Localization import Localization
# except:
# from KITT_Control.Localization import Localization
# pass
class sounddevice:
def __init__(self, x=240, y=30, theta=np.pi/2):
self.state = SharedState(x , y, theta)
self.state.beacon = True # Turn on the beacon for testing!!
def get_device_count(self):
return 5
def get_device_info_by_index(self, index):
example_output = [
{'index': 0, 'structVersion': 2, 'name': 'iPhone Microphone', 'hostApi': 0, 'maxInputChannels': 1, 'maxOutputChannels': 0, 'defaultLowInputLatency': 0.12841666666666668, 'defaultLowOutputLatency': 0.01, 'defaultHighInputLatency': 0.13775, 'defaultHighOutputLatency': 0.1, 'defaultSampleRate': 48000.0},
{'index': 1, 'structVersion': 2, 'name': 'Scarlett 18i20', 'hostApi': 0, 'maxInputChannels': 18, 'maxOutputChannels': 18, 'defaultLowInputLatency': 0.01, 'defaultLowOutputLatency': 0.0045578231292517, 'defaultHighInputLatency': 0.1, 'defaultHighOutputLatency': 0.014716553287981859, 'defaultSampleRate': 48000.0},
{'index': 2, 'structVersion': 2, 'name': 'MacBook Pro Microphone', 'hostApi': 0, 'maxInputChannels': 1, 'maxOutputChannels': 0, 'defaultLowInputLatency': 0.03285416666666666, 'defaultLowOutputLatency': 0.01, 'defaultHighInputLatency': 0.0421875, 'defaultHighOutputLatency': 0.1, 'defaultSampleRate': 48000.0},
{'index': 3, 'structVersion': 2, 'name': 'MacBook Pro Speakers', 'hostApi': 0, 'maxInputChannels': 0, 'maxOutputChannels': 2, 'defaultLowInputLatency': 0.01, 'defaultLowOutputLatency': 0.018708333333333334, 'defaultHighInputLatency': 0.1, 'defaultHighOutputLatency': 0.028041666666666666, 'defaultSampleRate': 48000.0},
{'index': 4, 'structVersion': 2, 'name': 'Microsoft Teams Audio', 'hostApi': 0, 'maxInputChannels': 2, 'maxOutputChannels': 2, 'defaultLowInputLatency': 0.01, 'defaultLowOutputLatency': 0.0013333333333333333, 'defaultHighInputLatency': 0.1, 'defaultHighOutputLatency': 0.010666666666666666, 'defaultSampleRate': 48000.0},
]
return example_output[index]
def open(self, input_device_index, channels, format, rate, input):
self.stream = Stream(self.state)
return self.stream
class Stream:
def __init__(self, state):
self.state = state
self.pulse = []
self.silence = []
# Load the recording
self.load_recordings()
self.Fs = 44100 # Sampling frequency
self.num_pulses = 3
self.speed_of_sound = 34300 # Speed of sound in cm/s
def load_recordings(self):
# Load the recordings from files
try:
with open("KITT_Simulator/simulator_data/pulses_recording.txt", "r") as f:
line = f.readline()
self.pulse = np.array([int(i) for i in line.strip().split()[1:]], dtype=np.float32)
with open("KITT_Simulator/simulator_data/silence.txt", "r") as f:
self.silence = np.array([int(i) for i in f.readline().strip().split()[1:]], dtype=np.float32)
except:
with open("simulator_data/pulses_recording.txt", "r") as f:
line = f.readline()
self.pulse = np.array([int(i) for i in line.strip().split()[1:]], dtype=np.float32)
with open("simulator_data/silence.txt", "r") as f:
self.silence = np.array([int(i) for i in f.readline().strip().split()[1:]], dtype=np.float32)
def read(self, num_frames):
overflow = False
distances = self.__dist()
distances = [dist * self.Fs / self.speed_of_sound for dist in distances] # Convert distances to samples
R1 = self.pulse[1000-int(distances[0]):]
R2 = self.pulse[1000-int(distances[1]):]
R3 = self.pulse[1000-int(distances[2]):]
R4 = self.pulse[1000-int(distances[3]):]
R5 = self.pulse[1000-int(distances[4]):]
R1 = R1[:33000]
R2 = R2[:33000]
R3 = R3[:33000]
R4 = R4[:33000]
R5 = R5[:33000]
buffer = self.__interleave_and_prepare_buffer(R1, R2, R3, R4, R5)
return buffer, overflow
def __dist(self):
# Calculate distances from the car to each microphone
mic_coor = np.array([[0, 0], [0, 460], [460, 460], [460, 0], [230, 0]])
car_coor = np.array([self.state.x, self.state.y])
D1 = np.linalg.norm(mic_coor[0] - car_coor)
D2 = np.linalg.norm(mic_coor[1] - car_coor)
D3 = np.linalg.norm(mic_coor[2] - car_coor)
D4 = np.linalg.norm(mic_coor[3] - car_coor)
D5 = np.linalg.norm(mic_coor[4] - car_coor)
return D1, D2, D3, D4, D5
@staticmethod
def __interleave_and_prepare_buffer(*arrays):
# Interleave arrays and prepare buffer
length = min(len(arr) for arr in arrays)
arrays = [arr[:length] for arr in arrays]
interleaved_list = np.vstack(arrays).reshape((-1,), order='F')
buffer = interleaved_list.astype(np.float32)
return buffer
if __name__ == "__main__":
sounddevice_handle = sounddevice(x=400, y=140, theta=np.pi/2)
for i in range(sounddevice_handle.get_device_count()):
device_info = sounddevice_handle.get_device_info_by_index(i)
print(i, device_info['name'])
device_index = int(input('Enter device index: '))
Fs = 44100
stream = sounddevice_handle.open(
input_device_index=device_index, channels=5, format='float32', rate=Fs, input=True
)
samples,_ = stream.read(Fs * 6)
data = np.frombuffer(samples, dtype='float32')
recording = np.array([data[0::5], data[1::5], data[2::5], data[3::5], data[4::5]]).T
plt.plot(recording)
plt.title("Recording")
plt.show()
# Proceed with localization using the corrected recordings
pulse_dict = {}
with open("KITT_Simulator/simulator_data/pulses_recording.txt", "r") as f:
for line in f:
line = line.strip().split()
pulse_dict[int(line[0][:-1])] = [int(i) for i in line[1:]]
refSignal = pulse_dict[30][16000:19500]
# localization = Localization(recording, refSignal, 3, Fs)
# Get the localized position
# x_car, y_car = localization.localizations
# print(f"Estimated position: x={x_car}, y={y_car}")

1
README.md Normal file
View File

@ -0,0 +1 @@
# Book

19
Student Code/control.py Normal file
View File

@ -0,0 +1,19 @@
class KITT:
def __init__(self, port, baudrate=115200):
self.serial = Serial(port, baudrate, rtscts=True)
def send_command(self, command):
self.serial.write(command.encode())
def set_speed(self, speed):
self.send_command(f'M{speed}\n')
def set_angle(self, angle):
self.send_command(f'D{angle}\n')
def stop(self):
self.set_speed(150)
self.set_angle(150)
def __del__(self):
self.serial.close()

97
appendix/0_Installation_Linux.ipynb Normal file
View File

@ -0,0 +1,97 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](../pictures/header.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"1. Open a terminal.\n",
" \n",
"2. Update your package manager's repository information:\n",
" \n",
"```{code-cell}\n",
" sudo apt update # For Debian/Ubuntu\n",
"``` \n",
"3. Install Python:\n",
" \n",
"```{code-cell}\n",
" sudo apt install python3 # For Debian/Ubuntu\n",
"```\n",
" \n",
"4. To verify the installation, type:\n",
" \n",
"```{code-cell}\n",
" python3 --version\n",
"```\n",
" \n",
"This should display the installed Python version."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Open a terminal in your IP3 directory and run the following command:\n",
"```{code-cell}\n",
"pip3 install -r requirements.txt\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 1. **Linux and macOS (Intel-based)**:\n",
"On Linux and macOS (except for ARM versions like Apple Silicon), Sounddevice works out of the box. Follow these steps to ensure it’s correctly installed:\n",
"\n",
"- First, install Sounddevice using `pip`:\n",
"\n",
"```bash\n",
"pip3 install sounddevice\n",
"```\n",
"\n",
"#### 2. **macOS (Apple Silicon - ARM)**:\n",
"For users on Apple Silicon (M1, M2), the ARM version of macOS, Sounddevice needs to be installed via **Homebrew**. Follow these steps:\n",
"\n",
"- First, install the PortAudio library (which Sounddevice depends on) using Homebrew:\n",
"\n",
"```bash\n",
"brew install portaudio\n",
"```\n",
"\n",
"- Then, install Sounddevice using `pip`:\n",
"\n",
"```bash\n",
"pip3 install sounddevice\n",
"```"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "OpenEdVenv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

361
appendix/0_Installation_Mac.ipynb Normal file
View File

@ -0,0 +1,361 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](../pictures/header.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Installation Guide for macOS\n",
"\n",
"This guide will walk you through installing Python, Visual Studio Code, and all the necessary dependencies for your project.\n",
"\n",
"At the bottom of this guide, you will find a section on common problems and solutions. If you encounter any issues during the installation process, please refer to that section for troubleshooting tips.\n",
"\n",
"## Installing Homebrew\n",
"\n",
"Homebrew is a package manager for macOS that simplifies the installation of software. We will use Homebrew to install Python and other dependencies.\n",
"\n",
"1. **Install Homebrew**:\n",
"\n",
" - Searche **\"Terminal\"** in Spotlight.\n",
" - Run the following command:\n",
"\n",
" ```bash\n",
" /bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"\n",
" ```\n",
"\n",
" - You will be prompted for your password. Enter it to proceed with the installation.\n",
" - If prompted with follow-up commands, run them as instructed.\n",
"\n",
"2. **Verify Homebrew Installation**:\n",
"\n",
" - After installation, verify that Homebrew is installed correctly:\n",
"\n",
" ```bash\n",
" brew --version\n",
" ```\n",
"\n",
" - This should display the installed Homebrew version.\n",
"\n",
"## Installing Python\n",
"\n",
"With Homebrew installed, we can now install Python **3.12** using Homebrew.\n",
"\n",
"1. **Install Python 3.12**:\n",
"\n",
" - In the Terminal, run:\n",
"\n",
" ```bash\n",
" brew install python@3.12\n",
" ```\n",
"\n",
" - This will install Python 3.12 and make it available as `python3`.\n",
"\n",
"2. **Verify the Installation**:\n",
"\n",
" ```bash\n",
" python3 --version\n",
" ```\n",
"\n",
" - This should display the installed Python version, e.g., `Python 3.12.0`.\n",
" - **Note**: On macOS, `python` may point to Python 2.x. Always use `python3` and `pip3` to invoke Python 3.\n",
"\n",
"3. **Update PATH (if necessary)**:\n",
"\n",
" - If you encounter issues with the `python3` command, you may need to update your PATH.\n",
" - Follow Homebrew's post-installation messages, which may prompt you to add the following to your shell profile (`~/.zprofile` or `~/.bash_profile`):\n",
"\n",
" ```bash\n",
" echo 'export PATH=\"/usr/local/opt/python@3.12/bin:$PATH\"' >> ~/.zprofile\n",
" ```\n",
"\n",
" - Reload your shell configuration:\n",
"\n",
" ```bash\n",
" source ~/.zprofile\n",
" ```\n",
"\n",
"## Installing Visual Studio Code\n",
"\n",
"It is recommended that you use Visual Studio Code (VSCode) as your Integrated Development Environment (IDE). VSCode is a free, open-source code editor developed by Microsoft with a wide range of extensions and excellent Python support.\n",
"\n",
"1. **Download VSCode**:\n",
"\n",
" - Open your web browser and navigate to the official VSCode website at [https://code.visualstudio.com/](https://code.visualstudio.com/).\n",
"\n",
" - Click on the \"**Download**\" button to download the installer suitable for your operating system.\n",
"\n",
"2. **Install VSCode**:\n",
"\n",
" - Once the installer is downloaded, run it to start the installation process.\n",
"\n",
"3. **Launch VSCode**:\n",
"\n",
" - After the installation is complete, open Visual Studio Code.\n",
"\n",
"### Installing Python Extension for VSCode\n",
"\n",
"1. **Open VSCode**.\n",
"\n",
"2. **Access the Extensions View**:\n",
"\n",
" - Click on the Extensions icon in the Activity Bar on the side of the window.\n",
" - Or press **Command+Shift+X**.\n",
"\n",
"3. **Install the Python Extension**:\n",
"\n",
" - In the Extensions search bar, type **\"Python\"**.\n",
" - Look for the official Python extension by Microsoft and click **\"Install\"**.\n",
" - This extension provides enhanced support for Python development.\n",
"\n",
"4. **Install the Jupyter Extension**:\n",
"\n",
" - In the same way, search for **\"Jupyter\"**.\n",
" - Install the official Jupyter extension by Microsoft.\n",
"\n",
"5. **Reload VSCode**:\n",
"\n",
" - After installation, you may need to reload VSCode to activate the extensions fully.\n",
" - If prompted, click on the **\"Reload\"** button.\n",
"\n",
"6. **Select the Python Interpreter**:\n",
"\n",
" - Open the Command Palette by pressing **Command+Shift+P**.\n",
" - Type **\"Python: Select Interpreter\"** and select the Python 3.12 interpreter from the list.\n",
" - If you don't see it, click on **\"Enter interpreter path...\"** and navigate to `/usr/local/bin/python3`.\n",
"\n",
"## Cloning the Git Repository\n",
"\n",
"If you already have a way to work with Git repositories, you can skip to step 3. If you don't, follow these steps to clone the repository to your computer using VSCode's built-in Git support.\n",
"\n",
"1. **Open the Source Control View in VSCode**:\n",
"\n",
" - Click on the **Source Control** icon in the Activity Bar.\n",
"\n",
" - Or press **Control+Shift+G**.\n",
"\n",
"2. **Install Git (if not already installed)**:\n",
"\n",
" - If you don't have Git installed, VSCode will display a message and a **\"Download Git\"** button.\n",
"\n",
" - Click the button or in the Terminal, run:\n",
"\n",
" ```bash\n",
" brew install git\n",
" ```\n",
"\n",
"3. **Configure Git (First-Time Setup)**:\n",
"\n",
" - Set your Git user name and email:\n",
"\n",
" ```bash\n",
" git config --global user.name \"Your Name\"\n",
" git config --global user.email \"you@example.com\"\n",
" ```\n",
"\n",
"4. **Clone the Repository**:\n",
"\n",
" - Go to your assigned GitLab repository from the [EWI GitLab](https://gitlab.ewi.tudelft.nl/).\n",
" - Click the blue **\"Clone\"** or **\"Code\"** button and copy the HTTPS URL.\n",
"\n",
"5. **Clone Using VSCode**:\n",
"\n",
" - In VSCode, go to the **Source Control** view by clicking on the Source Control icon in the Activity Bar or by pressing **Command+Shift+G**.\n",
" - Click the **\"Clone Repository\"** button.\n",
" - Paste the URL you just copied and press **Enter**.\n",
" - Choose a folder to download the project to; make it something memorable.\n",
" - When prompted, open the newly cloned repository.\n",
" - If asked to trust the authors, please do so.\n",
"\n",
"## Installing Required Packages\n",
"\n",
"1. **Install PortAudio**:\n",
"\n",
" - In the Terminal, run:\n",
"\n",
" ```bash\n",
" brew install portaudio\n",
" ```\n",
"\n",
" - This installs PortAudio, a prerequisite for Sounddevice, which is used for audio input/output.\n",
"\n",
"2. **Install the Required Python Packages**:\n",
"\n",
" - Open a Terminal in your project directory (the cloned repository).\n",
" - Run the following command:\n",
"\n",
" ```bash\n",
" pip3 install -r requirements.txt\n",
" ```\n",
"\n",
" - **Note**: Always use `pip3` and `python3` on macOS to ensure you're using the correct version.\n",
"\n",
"---\n",
"\n",
"## Common Problems and Solutions\n",
"\n",
"### Problem 1: `'python3' command not found`\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: The `python3` command is not in your PATH.\n",
"- **Fix**:\n",
" - Ensure that Python 3.12 is installed via Homebrew.\n",
" - Update your PATH environment variable by adding the following to your shell profile (`~/.zprofile`, `~/.bash_profile`, or `~/.bashrc`):\n",
"\n",
" ```bash\n",
" echo 'export PATH=\"/usr/local/opt/python@3.12/bin:$PATH\"' >> ~/.zprofile\n",
" ```\n",
"\n",
" - Reload your shell configuration:\n",
"\n",
" ```bash\n",
" source ~/.zprofile\n",
" ```\n",
"\n",
"### Problem 2: `'pip3' command not found`\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: `pip3` may not be linked correctly.\n",
"- **Fix**:\n",
" - Use `python3 -m pip` instead of `pip3`:\n",
"\n",
" ```bash\n",
" python3 -m pip install -r requirements.txt\n",
" ```\n",
"\n",
"- **Ensure `pip` is up to date**:\n",
"\n",
" ```bash\n",
" python3 -m pip install --upgrade pip\n",
" ```\n",
"\n",
"### Problem 3: Homebrew Installation Issues\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: Homebrew may not install correctly due to existing installations or permissions.\n",
"- **Fix**:\n",
" - Uninstall any existing Homebrew installations:\n",
"\n",
" ```bash\n",
" /bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/uninstall.sh)\"\n",
" ```\n",
"\n",
" - Reinstall Homebrew.\n",
"\n",
"### Problem 4: Permission Denied Errors When Installing Packages\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: Lack of permissions to write to certain directories.\n",
"- **Fix**:\n",
" - Install packages with the `--user` flag:\n",
"\n",
" ```bash\n",
" pip3 install --user -r requirements.txt\n",
" ```\n",
"\n",
" - Alternatively, use a virtual environment:\n",
"\n",
" ```bash\n",
" pip3 install virtualenv\n",
" python3 -m venv venv\n",
" source venv/bin/activate\n",
" pip install -r requirements.txt\n",
" ```\n",
"\n",
"\n",
"### Problem 5: Git Command Not Found\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: Git is not installed or not in your PATH.\n",
"- **Fix**:\n",
" - Install Git using Homebrew:\n",
"\n",
" ```bash\n",
" brew install git\n",
" ```\n",
"\n",
" - Verify the installation:\n",
"\n",
" ```bash\n",
" git --version\n",
" ```\n",
"\n",
" - If Git is still not found, update your PATH:\n",
"\n",
" ```bash\n",
" echo 'export PATH=\"/usr/local/opt/git/bin:$PATH\"' >> ~/.zprofile\n",
" source ~/.zprofile\n",
" ```\n",
"\n",
"### Problem 6: Authentication Issues When Cloning the Repository\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: Incorrect credentials or two-factor authentication.\n",
"- **Fix**:\n",
" - Ensure you're using the correct username and password.\n",
" - If using two-factor authentication, generate a Personal Access Token on GitLab and use it as your password.\n",
" - Consider setting up SSH keys for authentication.\n",
"\n",
"### Problem 7: Permission Denied When Running Scripts\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: Script files may not have execute permissions.\n",
"- **Fix**:\n",
" - Modify the file permissions:\n",
"\n",
" ```bash\n",
" chmod +x script.py\n",
" ```\n",
"\n",
" - Run the script using `python3`:\n",
"\n",
" ```bash\n",
" python3 script.py\n",
" ```\n",
"\n",
"---\n",
"\n",
"**Note**: If you encounter issues not covered in this guide, please reach out to a TA for assistance. Providing detailed error messages and the steps you've taken can help diagnose problems more quickly."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "OpenEdVenv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

500
appendix/0_Installation_Windows.ipynb Normal file
View File

@ -0,0 +1,500 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![book header](../pictures/header.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Installation Guide for Windows\n",
"\n",
"As everyone is transitioning to Windows 11, this installation guide focuses on setting up the environment on Windows 11. The guide is also applicable to Windows 10, but some steps may differ slightly; we have added differing instructions whenever required.\n",
"\n",
"At the bottom of this guide, you will find a section on common problems and solutions. If you encounter any issues during the installation process, please refer to that section for troubleshooting tips.\n",
"\n",
"## Installing Python\n",
"\n",
"The first step in setting up your Python environment is to install the Python interpreter. For the best compatibility, everyone on the team should install the same version of Python. As of 2024, it is recommended that Python 3.12 be installed. The KITT simulator requires a Python version of 3.10 or newer, so any version older than 3.10 is strongly discouraged.\n",
"\n",
"1. **Install Python from the Microsoft Store**:\n",
"\n",
" - Open the Microsoft Store and search for \"Python\". Install **Python 3.12** by clicking on it and then selecting \"Get\" or \"Install\".\n",
"\n",
" - **Note**: Installing from the Microsoft Store ensures automatic updates and easy management. However, if you prefer, you can download Python directly from the [official website](https://www.python.org/downloads/windows/).\n",
"\n",
"2. **Verify the Installation**:\n",
"\n",
" - Open a command prompt:\n",
"\n",
" - Press the **Windows key**, type `cmd`, and hit **Enter**.\n",
"\n",
" - Type the following command and press **Enter**:\n",
"\n",
" ```bash\n",
" python --version\n",
" ```\n",
"\n",
" - This should display the installed Python version, e.g., `Python 3.12.0`. Please verify that the installed version matches the one you downloaded.\n",
"\n",
" - **Troubleshooting**:\n",
"\n",
" - If you receive an error like `'python' is not recognized as an internal or external command`, it means Python is not added to your system's PATH environment variable.\n",
"\n",
" - Follow this tutorial to [add Python to PATH](https://realpython.com/add-python-to-path/), or refer to the **Common Problems and Solutions** section below.\n",
"\n",
" - **Multiple Python Versions**:\n",
"\n",
" - If you have multiple versions of Python installed, you may need to specify `python3` instead of `python`:\n",
"\n",
" ```bash\n",
" python3 --version\n",
" ```\n",
"\n",
"## Installing Visual Studio Code\n",
"\n",
"It is recommended that you use Visual Studio Code (VSCode) as your Integrated Development Environment (IDE). VSCode is a free, open-source code editor developed by Microsoft with a wide range of extensions and excellent Python support.\n",
"\n",
"1. **Download VSCode**:\n",
"\n",
" - Open your web browser and navigate to the official VSCode website at [https://code.visualstudio.com/](https://code.visualstudio.com/).\n",
"\n",
" - Click on the \"**Download**\" button to download the installer suitable for your operating system.\n",
"\n",
"2. **Install VSCode**:\n",
"\n",
" - Once the installer is downloaded, run it to start the installation process.\n",
"\n",
" - **Installation Options**:\n",
"\n",
" - You can choose the default settings unless you have specific preferences.\n",
"\n",
" - **Recommended**: Check the option to \"Add to PATH\" during installation to allow you to open files and folders from the command line.\n",
"\n",
"3. **Launch VSCode**:\n",
"\n",
" - After the installation is complete, open Visual Studio Code.\n",
"\n",
"### Installing Python Extension for VSCode\n",
"\n",
"VSCode is very popular because of its use of extensions. This makes it compatible with almost any programming language or development board (such as Arduino). In our case, you have to add Python support to the IDE. Follow these steps:\n",
"\n",
"1. **Open VSCode**.\n",
"\n",
"2. **Access the Extensions View**:\n",
"\n",
" - Click on the Extensions icon in the Activity Bar (left side of the screen, a stack of blocks).\n",
"\n",
" - Or press **Control+Shift+X**.\n",
"\n",
"3. **Install the Python Extension**:\n",
"\n",
" - In the Extensions search bar, type **\"Python\"**.\n",
"\n",
" - Look for the official Python extension by Microsoft and click **\"Install\"**. This extension provides enhanced support for Python development, including IntelliSense, linting, debugging, and more.\n",
"\n",
"4. **Install the Jupyter Extension**:\n",
"\n",
" - In the same way, search for **\"Jupyter\"**.\n",
"\n",
" - Install the official Jupyter extension by Microsoft.\n",
"\n",
"5. **Reload VSCode**:\n",
"\n",
" - After installation, you may need to reload VSCode to activate the extensions fully.\n",
"\n",
" - If prompted, click on the **\"Reload\"** button.\n",
"\n",
"6. **Select the Python Interpreter**:\n",
"\n",
" - Open the Command Palette by pressing **Control+Shift+P**.\n",
"\n",
" - Type **\"Python: Select Interpreter\"** and select the Python 3.12 interpreter from the list.\n",
"\n",
" - If you don't see it, click on **\"Enter interpreter path\"** and navigate to your Python installation directory.\n",
"\n",
"## Cloning the Git Repository\n",
"\n",
"If you already have a way to work with Git repositories, you can skip to step 3. If you don't, follow these steps to clone the repository to your computer using VSCode's built-in Git support.\n",
"\n",
"1. **Open the Source Control View in VSCode**:\n",
"\n",
" - Click on the **Source Control** icon in the Activity Bar.\n",
"\n",
" - Or press **Control+Shift+G**.\n",
"\n",
"2. **Install Git (if not already installed)**:\n",
"\n",
" - If you don't have Git installed, VSCode will display a message and a **\"Download Git\"** button.\n",
"\n",
" - Click the button or download Git from [https://git-scm.com/download/win](https://git-scm.com/download/win).\n",
"\n",
" - **Git Installation Options**:\n",
"\n",
" - Run the installer and accept the default settings, except for the **default behavior of `git pull`**:\n",
"\n",
" - When prompted, select **\"Rebase\"** as the default behavior for `git pull`. This makes merging commits easier and keeps your commit history cleaner.\n",
"\n",
" - Ensure that the option **\"Use Git from the Windows Command Prompt\"** is selected to add Git to your system PATH.\n",
"\n",
" - **Restart VSCode** after installing Git to ensure it recognizes the Git installation.\n",
"\n",
"3. **Configure Git (First-Time Setup)**:\n",
"\n",
" - Set your Git user name and email:\n",
"\n",
" ```bash\n",
" git config --global user.name \"Your Name\"\n",
" git config --global user.email \"you@example.com\"\n",
" ```\n",
"\n",
"4. **Clone the Repository**:\n",
"\n",
" - Go to your assigned GitLab repository from the [EWI GitLab](https://gitlab.ewi.tudelft.nl/). Click to sign in with `TU Delft SSO NetID`. Browse to your IP3 project repository.\n",
"\n",
" - Click the blue **\"Clone\"** or **\"Code\"** button and copy the HTTPS URL.\n",
"\n",
"5. **Clone in VSCode**:\n",
"\n",
" - In the Source Control view of VSCode, click on **\"Clone Repository\"**.\n",
"\n",
" - Paste the URL you just copied and press **Enter**.\n",
"\n",
" - **Authentication**:\n",
"\n",
" - If prompted, enter your GitLab username and password.\n",
"\n",
" - If your account uses two-factor authentication, you may need to use a Personal Access Token instead of your password.\n",
"\n",
" - Choose a folder to download the project to.\n",
"\n",
"6. **Open the Cloned Repository**:\n",
"\n",
" - When asked, click **\"Open\"** to open the newly cloned repository.\n",
"\n",
" - If you're asked to trust the authors, please do so by clicking **\"Yes, I trust the authors\"**.\n",
"\n",
"## Installing Required Packages\n",
"\n",
"Python is a well-supported language with many possibilities. These possibilities come from the many libraries and packages that are built by users. Because we do not want to rewrite all the essentials, we will use a list of required packages that every user has to install.\n",
"\n",
"1. **Install the Microsoft Visual C++ Redistributable** (Windows 10 only):\n",
"\n",
" - Download and install the [Microsoft Visual C++ Redistributable](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist) if you haven't already.\n",
"\n",
" - This is required for compiling some Python packages.\n",
"\n",
"2. **Open the Project in VSCode**:\n",
"\n",
" - Open VSCode and navigate to the cloned repository.\n",
"\n",
"3. **Open a Terminal in VSCode**:\n",
"\n",
" - From the top menu, select **\"View\" > \"Terminal\"**.\n",
"\n",
" - Or press **Control+`** (the backtick key).\n",
"\n",
" - Ensure that the terminal is open in the root directory of your cloned repository (where the `requirements.txt` file is located).\n",
"\n",
"4. **Install the Required Packages**:\n",
"\n",
" - Run the following command:\n",
"\n",
" ```bash\n",
" pip install -r requirements.txt\n",
" ```\n",
" ```\n",
"\n",
"## Installing Sound Card Drivers\n",
"\n",
"For most audio measurements, we will be using the [Focusrite Scarlett 18i20](https://focusrite.com/products/scarlett-18i20) USB soundcard, shown in Fig. 1. \n",
"\n",
"It has 8 analog inputs (ADCs) and 10 analog outputs (DACs). \n",
"You connect it to your laptop/PC via a USB plug. \n",
"\n",
"---\n",
"\n",
"### Focusrite soundcard\n",
"\n",
"**(a)** \n",
"<img src=\"focusrite_front.JPG\" alt=\"Focusrite front\" width=\"60%\">\n",
"\n",
"**(b)** \n",
"<img src=\"focusrite_back.JPG\" alt=\"Focusrite back\" width=\"60%\">\n",
"\n",
"**Figure 1:** Focusrite soundcard — (a) front, and (b) backside. \n",
"\n",
"---\n",
"\n",
"The installation description given below has been checked for the **Windows 10** operating system. \n",
"It is expected that it is similar for **Windows 11** and for **Mac** computers. \n",
"For **Linux** machines, no driver installation is required. \n",
"\n",
"---\n",
"\n",
"### Setting up the control software\n",
"\n",
"When the soundcard is connected via USB, it will create a drive with a link to the Focusrite website. \n",
"From there you can download the proper Focusrite software package **“Focusrite Control”** for Windows or Mac. \n",
"\n",
"If this does not work, you can download the driver directly from: \n",
"[https://downloads.focusrite.com/focusrite/scarlett-3rd-gen/scarlett-18i20-3rd-gen](https://downloads.focusrite.com/focusrite/scarlett-3rd-gen/scarlett-18i20-3rd-gen) \n",
"\n",
"After installation, the following drivers are available: \n",
"- Windows driver \n",
"- Proprietary Focusrite ASIO driver \n",
"- Focusrite Control panel \n",
"\n",
"---\n",
"\n",
"Start *Focusrite Control* from your desktop. \n",
"\n",
"From the **Device** tab: \n",
"- Choose the sample rate and select **48 kHz**. \n",
"\n",
"Don’t change anything in the **Input settings**, i.e., Analog 1 and 2 are set to *Line* and the rest are off. \n",
"In **Output Routing**, the playback channels are routed to Outputs 1 + 2 — no change is needed. \n",
"\n",
"---\n",
"\n",
"In the Windows sound settings, you will initially see only two input channels: **Analog 1 + 2**. \n",
"To see all 8 analog input channels: \n",
"\n",
"1. Go to the **hidden icons** panel (bottom bar, at right). \n",
"2. Select the **“F”** icon and choose **Expose/Hidden Windows Channels**. \n",
"3. Check the boxes for **Analog 3 + 4**, **5 + 6**, and **7 + 8**. \n",
"\n",
"---\n",
"\n",
"Now open the soundcard settings panel: \n",
"**Start Menu → Control Panel → Sound → Recording**. \n",
"\n",
"Set **Analog 1 + 2** as *default*. Then: \n",
"\n",
"1. Select **Analog 1 + 2** and click **Properties**. \n",
"2. Under **Advanced**, choose **8 channel, 24 bit, 48000 Hz (Studio Quality)** as the *Default Format*. \n",
" - This allows recording from all 8 analog input channels in parallel. \n",
"3. Leave the *Advanced* window, go to **Level**, and make sure the recording level is set to **100 %**. \n",
"\n",
"---\n",
"\n",
"### Drivers\n",
"\n",
"For recording the eight input signals (**Analog 1 – 8**) simultaneously, the **Focusrite ASIO driver** must be used. \n",
"\n",
"A different ASIO driver (e.g., **ASIO4ALL**) will **not** work and must be removed. \n",
"\n",
"\n",
"## Common Problems and Solutions\n",
"\n",
"### Problem 1: `'python' is not recognized as an internal or external command`\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: Python is not added to your system's PATH environment variable.\n",
"\n",
"- **Fix**:\n",
"\n",
" 1. **Add Python to PATH**:\n",
"\n",
" - Go to **Control Panel** > **System and Security** > **System** > **Advanced system settings**.\n",
"\n",
" - Click on **\"Environment Variables\"**.\n",
"\n",
" - Under **\"System variables\"**, find **\"Path\"** and click **\"Edit\"**.\n",
"\n",
" - Click **\"New\"** and add the path to your Python installation (e.g., `C:\\Users\\<YourUsername>\\AppData\\Local\\Programs\\Python\\Python312\\`).\n",
"\n",
" 2. **Reinstall Python with PATH Option**:\n",
"\n",
" - Re-run the Python installer.\n",
"\n",
" - Check the option that says **\"Add Python 3.12 to PATH\"**.\n",
"\n",
" 3. **Verify**:\n",
"\n",
" - Open a new command prompt and type `python --version`.\n",
"\n",
"### Problem 2: `'pip' is not recognized as an internal or external command`\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: `pip` is not added to your system's PATH or not installed.\n",
"\n",
"- **Fix**:\n",
"\n",
" 1. **Use Python to Access pip**:\n",
"\n",
" ```bash\n",
" python -m pip install --upgrade pip\n",
" ```\n",
"\n",
" 2. **Ensure pip is Installed**:\n",
"\n",
" - Download `get-pip.py` from [https://bootstrap.pypa.io/get-pip.py](https://bootstrap.pypa.io/get-pip.py).\n",
"\n",
" - Run the script:\n",
"\n",
" ```bash\n",
" python get-pip.py\n",
" ```\n",
"\n",
"### Problem 3: Multiple Python Versions Causing Confusion\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: Multiple Python installations can lead to conflicts.\n",
"\n",
"- **Fix**:\n",
"\n",
" 1. **Check Python Installations**:\n",
"\n",
" ```bash\n",
" where python\n",
" ```\n",
"\n",
" - This will list all Python executables in your PATH.\n",
"\n",
" 2. **Specify Full Path**:\n",
"\n",
" - Use the full path to the desired Python version:\n",
"\n",
" ```bash\n",
" \"C:\\Users\\<YourUsername>\\AppData\\Local\\Programs\\Python\\Python312\\python.exe\" --version\n",
" ```\n",
"\n",
" 3. **Adjust PATH Environment Variable**:\n",
"\n",
" - Ensure that the path to Python 3.12 is higher in the PATH order.\n",
"\n",
"### Problem 4: Git is Not Recognized or Installation Issues\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: Git is not installed or not added to PATH.\n",
"\n",
"- **Fix**:\n",
"\n",
" 1. **Install Git**:\n",
"\n",
" - Download from [https://git-scm.com/download/win](https://git-scm.com/download/win).\n",
"\n",
" - During installation, select **\"Use Git from the Windows Command Prompt\"**.\n",
"\n",
" 2. **Verify Git Installation**:\n",
"\n",
" ```bash\n",
" git --version\n",
" ```\n",
"\n",
" 3. **Restart VSCode** to recognize the Git installation.\n",
"\n",
"### Problem 5: Errors When Installing Packages from `requirements.txt`\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: Missing dependencies or incompatible package versions.\n",
"\n",
"- **Fix**:\n",
"\n",
" 1. **Read Error Messages Carefully**:\n",
"\n",
" - Identify which package is causing the issue.\n",
"\n",
" 2. **Install Microsoft Visual C++ Redistributable**:\n",
"\n",
" - Download from [Microsoft's website](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist).\n",
"\n",
" 3. **Install Packages Individually**:\n",
"\n",
" - Remove the problematic package from `requirements.txt`.\n",
"\n",
" - Try installing problematic packages one by one for detailed errors:\n",
"\n",
" ```bash\n",
" pip install package_name\n",
" ```\n",
"\n",
" - Ask a TA for help if needed.\n",
"\n",
"\n",
"### Problem 6: VSCode Does Not Detect the Python Interpreter\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: VSCode can't find the Python installation.\n",
"\n",
"- **Fix**:\n",
"\n",
" 1. **Select Interpreter Manually**:\n",
"\n",
" - Press **Control+Shift+P** to open the Command Palette.\n",
"\n",
" - Type **\"Python: Select Interpreter\"**.\n",
"\n",
" - Choose the correct Python 3.12 interpreter.\n",
"\n",
" 2. **Install the Python Extension**:\n",
"\n",
" - Ensure the official Python extension by Microsoft is installed.\n",
"\n",
" 3. **Restart VSCode**.\n",
"\n",
"### Problem 7: VSCode Does Not Detect Git\n",
"\n",
"**Solution**:\n",
"\n",
"- **Cause**: Git is not installed or not in PATH.\n",
"\n",
"- **Fix**:\n",
"\n",
" 1. **Install Git** if not already installed.\n",
"\n",
" 2. **Add Git to PATH**:\n",
"\n",
" - Ensure that the Git installation directory is added to your system's PATH environment variable.\n",
"\n",
" - Go to: `Control Panel` > `System and Security` > `System` > `Advanced system settings` > `Environment Variables`.\n",
"\n",
" - Under **\"System variables\"**, find **\"Path\"** and click **\"Edit\"**.\n",
"\n",
" - Click **\"New\"** and add the path to your Git installation (e.g., `C:\\Program Files\\Git\\bin`).\n",
"\n",
" 3. **Configure Git Path in VSCode**:\n",
"\n",
" - Go to **\"Settings\"** in VSCode.\n",
"\n",
" - Search for **\"git.path\"** and set it to the path of your `git.exe` (e.g., `C:\\Program Files\\Git\\bin\\git.exe`).\n",
"\n",
"---\n",
"\n",
"**Note**: If you encounter issues not covered in this guide, please reach out to a TA for assistance. Providing screenshots of error messages can help diagnose problems more quickly."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "OpenEdVenv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

66
appendix/Appendix_A.ipynb Normal file
View File

@ -0,0 +1,66 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Appendix A: Programming the Audio Beacon\n",
"\n",
" This appendix describes the audio beacon signal parameters. The signal is generated by the M0 microcontroller of the LPC4357 ARM, an amplifier circuit (LM4752) and a loudspeaker. The microcontroller runs a dedicated program that produces continuously this signal on one of its PWM output pins which is amplified by an op-amp circuit and made audible by the loudspeaker.\n",
"\n",
"The generated signal is similar to the audio signals you have used in the EE2T11 Telecommunication A practicum (Labday 4). The parameters that define the signal can be set through the command interface; in contrast to the earlier practicum, now it is possible to use an arbitrary carrier frequency, bit frequency, and repetition count.\n",
"\n",
"## Audio Beacon signal parameters\n",
"\n",
"The audio beacon transmits a binary code sequence using ``on-off keying'' (OOK). If a bit in the sequence is 0, nothing is transmitted; if the bit is 1, a modulation carrier frequency is transmitted during a certain period. The number of bits is fixed at 32.\n",
"\n",
"Besides the actual bit sequence (a 32 bit code word), the parameters that determine the signal are:\n",
"\n",
"- Modulation carrier frequency (parameter *Freq0* specified in Hz), at most 30 kHz, although this is probably beyond the specs of the loudspeaker and microphones; \n",
"- Bit freqeuncy (parameter *Freq1* specified in Hz), this defines the rate at which the modulation carrier signal is switched on or off by bits in the code word, i.e., determines the duration of a single bit (and indirectly the bandwidth of the generated signal).\n",
"- Repetition count of the bit sequence (parameter *Count3*, an integer), this specifies the number of bits the beacon waits before transmitting the code again. The minimum value is 32 (otherwise a new code is transmitted before the previous one is finished). The resulting repetition frequency is, \n",
"\n",
"<p align=\"center\">\n",
" <img src=\"https://latex.codecogs.com/svg.latex?\\color{red}{repetition\\_frequency=\\frac{bit\\_frequency}{repetition\\_count}}\" alt=\"Equation\">\n",
"</p>\n",
"\n",
"For example, with a value of 5 kHz of the bit frequency, a repition count of 2500 gives a repetition count of 2500 gives a repetition freqeuncy of 2 kHz. You will probably want to have a higher repetition freqeuncy.\n",
"\n",
"[An example of pulses generated by the audio beacon](figaudiocode.pdf)\n",
"\n",
"\n",
"These parameters are set by sending them to the microcontroller on the car over the Bluetooth interface. The corresponding commands are described in *Controlling KITT* section in Module 1.\n",
"\n",
"The model *refsignal* that is used EE2T11 Telecommunications A practicum has similar parameters, but were limited to a specific set of values.\n",
"\n",
"The default setting of the audio beacon is a 32 bit code sequence bit-stream with:\n",
"\n",
"<center>\n",
"\n",
"| | | | | |\n",
"|-------|--------------------|---|-------------|----------|\n",
"| Freq0 | Carrier frequency | = | 15000 | Hz |\n",
"| Freq1 | Bit frequency | = | 5000 | Hz |\n",
"| Count3| Repeat counter | = | 64 | samples |\n",
"| | Code word | = | `0x92340f0f`| (hex) |\n",
"\n",
"</center>\n",
"\n",
"There is not guarantee at all this default setting is of high quality!\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

37
appendix/Appendix_B.ipynb Normal file
View File

@ -0,0 +1,37 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Appendix B: Serial Communication with Windows\n",
"\n",
"This Appendix describes how the serial connection to the car from Windows (using Bluetooth) is implemented.\n",
"\n",
"## Connecting KITT to a PC on Windows 10\n",
"\n",
"In the figure below, the steps can be seen how KITT can be connected to a PC with Windows 10 installed. \n",
"\n",
"- Note that all KITTs have their Bluetooth hardware address written in the front of the car. Use this identifier to choose the right Bluetooth device to pair: Step 3 and 4.\n",
"\n",
"- Step 6, the correct port to take note of is the one listed as *Outgoing*. This is the port you have to use in your setup.\n",
"\n",
"### Steps how to connect KITT to the PC\n",
"\n",
"![Steps how to connect KITT to PC](steps.png)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

104
appendix/Appendix_C.ipynb Normal file
View File

@ -0,0 +1,104 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Appendix C: TDOA Localization Algorithm\n",
"<a id=\"app-locating-algorithms\"></a>\n",
"\n",
"## Time-Difference Of Arrival (TDOA) Algorithm\n",
"<a id=\"sec-loc-tdoalocalization\"></a>\n",
"\n",
"Let $\\mathbf{x} = [x,y]^T$ be the location of the car, and call $\\mathbf{x}_i = [x_i,y_i]^T$, $i = 1, \\cdots, N$ the known locations of the microphones. The Time Difference of Arrival (TDOA) for each sensor pair $(i,j)$ is translated to a range difference (by multiplying by the speed of sound) $r_{ij}$, where\n",
"\n",
"$\n",
"r_{ij} = d_i - d_j\\,,\\qquad d_i = \\|\\mathbf{x} - \\mathbf{x}_i\\|\\,,\\qquad d_j = \\|\\mathbf{x} - \\mathbf{x}_j\\|\n",
"$\n",
"\n",
"The objective is to compute from the available measurements, $\\{r_{ij};\\; i,j = 1, \\cdots, N, i<j\\}$ the location $\\mathbf{x}$ of the car. Each range measurement specifies a hyperbolic curve in the 2-D plane of possible locations $\\mathbf{x}$, and the combination of measurements asks for the intersection of the curves (this is called *multilateration*). Equivalently, we have to solve a system of quadratic equations.\n",
"\n",
"Fortunately, it is possible to transform these into a system of *linear* equations, augmented with some additional \"nuisance\" parameters, as follows. Write\n",
"\n",
"\n",
"$r_{ij} = d_i - d_j \n",
"\\quad \\Rightarrow \\quad \n",
"(r_{ij} + d_j)^2 = d_i^2\n",
"\\quad \\Leftrightarrow \\quad \n",
"r_{ij}^2 + d_j^2 + 2 r_{ij}d_j = d_i^2$\n",
"\n",
"\n",
"We will use\n",
"\n",
"\n",
"$\\begin{aligned}\n",
"d_i^2 &= \\|\\mathbf{x} - \\mathbf{x}_i\\|^2 = \\|\\mathbf{x}\\|^2 + \\|\\mathbf{x}_i\\|^2 - 2\\mathbf{x}_i^T \\mathbf{x} \\\\\n",
"\\text{and}\\quad d_j^2 &= \\|\\mathbf{x} - \\mathbf{x}_j\\|^2 = \\|\\mathbf{x}\\|^2 + \\|\\mathbf{x}_j\\|^2 - 2\\mathbf{x}_j^T \\mathbf{x}\n",
"\\end{aligned}$\n",
"\n",
"\n",
"Inserting this into the previous equation, we see that the quadratic term $\\|\\mathbf{x}\\|^2$ is eliminated:\n",
"\n",
"$\n",
"\\begin{aligned}\n",
"r_{ij}^2 + d_j^2 & + 2 r_{ij}d_j = d_i^2 \\\\\n",
"&\\Leftrightarrow \\quad r_{ij}^2 + \\|\\mathbf{x}\\|^2 + \\|\\mathbf{x}_j\\|^2 - 2\\mathbf{x}_j^T \\mathbf{x} + 2 r_{ij} d_j \\\\\n",
"& \\qquad \\qquad = \\|\\mathbf{x}\\|^2 + \\|\\mathbf{x}_i\\|^2 - 2 \\mathbf{x}_i^T \\mathbf{x} \\\\\n",
"&\\Leftrightarrow \\quad r_{ij}^2 - \\|\\mathbf{x}_i\\|^2 + \\|\\mathbf{x}_j\\|^2 = 2 (\\mathbf{x}_j - \\mathbf{x}_i)^T \\mathbf{x} - 2 r_{ij} d_j\n",
"\\end{aligned}\n",
"$\n",
"\n",
"This can be written in vector notation as\n",
"\n",
"$\n",
"[2(\\mathbf{x}_j - \\mathbf{x}_i)^T \\quad -2r_{ij}] \\begin{bmatrix} \\mathbf{x} \\\\ d_j \\end{bmatrix} = r_{ij}^2 - \\|\\mathbf{x}_i\\|^2 + \\|\\mathbf{x}_j\\|^2\n",
"$\n",
"\n",
"where the row vector and RHS are known, and $\\mathbf{x}$ and $d_j$ are unknown. If we now write the equations for all pairs $(i,j)$, $i<j$ and assume $N=4$ sensors, we obtain a matrix equation as\n",
"\n",
"\n",
"\\begin{bmatrix}\n",
"2(\\mathbf{x}_2 - \\mathbf{x}_1)^T & -2r_{12} & & \\\\\n",
"2(\\mathbf{x}_3 - \\mathbf{x}_1)^T & & -2r_{13} & \\\\\n",
"2(\\mathbf{x}_4 - \\mathbf{x}_1)^T & & & -2r_{14} \\\\\n",
"2(\\mathbf{x}_3 - \\mathbf{x}_2)^T & & -2r_{23} & \\\\\n",
"2(\\mathbf{x}_4 - \\mathbf{x}_2)^T & & & -2r_{24} \\\\\n",
"2(\\mathbf{x}_4 - \\mathbf{x}_3)^T & & & -2r_{34} \n",
"\\end{bmatrix}\n",
"\\begin{bmatrix} \n",
"\\mathbf{x} \\\\ d_2 \\\\ d_3 \\\\ d_4 \n",
"\\end{bmatrix}\n",
"=\n",
"\\begin{bmatrix} \n",
"r_{12}^2 - \\|\\mathbf{x}_1\\|^2 + \\|\\mathbf{x}_2\\|^2 \\\\\n",
"r_{13}^2 - \\|\\mathbf{x}_1\\|^2 + \\|\\mathbf{x}_3\\|^2 \\\\\n",
"r_{14}^2 - \\|\\mathbf{x}_1\\|^2 + \\|\\mathbf{x}_4\\|^2 \\\\\n",
"r_{23}^2 - \\|\\mathbf{x}_2\\|^2 + \\|\\mathbf{x}_3\\|^2 \\\\\n",
"r_{24}^2 - \\|\\mathbf{x}_2\\|^2 + \\|\\mathbf{x}_4\\|^2 \\\\\n",
"r_{34}^2 - \\|\\mathbf{x}_3\\|^2 + \\|\\mathbf{x}_4\\|^2 \n",
"\\end{bmatrix}\n",
"\n",
"\n",
"This is an overdetermined linear set of equations (6 equations, 5 unknowns) of the form $\\mathbf{A} \\mathbf{y} = \\mathbf{b}$, and can be solved by computing the pseudo-inverse (or left-inverse) of $\\mathbf{A}$, i.e., $\\mathbf{y} = \\mathbf{A}^\\dagger \\mathbf{b} = (\\mathbf{A}^T\\mathbf{A})^{-1} \\mathbf{A}^T \\mathbf{b}$.\n",
"\n",
"We thus obtain $\\mathbf{x}$, as well as the nuisance parameters $d_2,d_3,d_4$ that depend on $\\mathbf{x}$.\n",
"\n",
"Although there are 6 equations, they are linearly dependent: verify this (e.g., on a noiseless testcase in Python).\n",
"\n",
"**Question:** What happens if $r_{12} = 0$?\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

214
appendix/Appendix_D.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

207
appendix/Controller.ipynb Normal file
View File

@ -0,0 +1,207 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Controller\n",
"**⚠️ Optional Advanced method**\n",
"The contents of this module cover advanced topics that not all students are expected to understand."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### The complex problem \n",
"\n",
"**Find a control law** $$ \\mathbf{u}(t) = \\begin{bmatrix} v(t) \\\\ \\phi(t) \\end{bmatrix} $$ which are the numbers that are sent to KITT as car velocity and steering that moves the car from the initial state $$ \\mathbf{x}(0) = \\mathbf{x}_0 $$ to the goal state $$ \\mathbf{x}(T) = \\mathbf{x}_g $$ in finite time T , while satisfying the following conditions:\n",
"\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"- **Kinematic Equations**: The car's motion is governed by equations that relate its position, orientation, velocity, and steering angle.\n",
"- **Non-Holonomic Constraint**: The car cannot move sideways; it can only move forward or backward along its current orientation.\n",
"- **Control Input Limits**:\n",
" - **Steering Angle Limit**: The steering angle cannot exceed a maximum absolute value.\n",
" - **Velocity Limit**: The car's speed must be within specified minimum and maximum values."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> **Simplifying the problem**\n",
">\n",
"> So as you see the control problem is quite complex. To make controlling the car from point A to point B manageable, we simplify the complex two-dimensional problem into a one-dimensional one. The idea is to do this by:\n",
">\n",
"> - **Planning a Feasible Path**: Create a path that the car can physically follow, considering its inability to move sideways and its steering limits.\n",
"> - **Projecting Movement onto the Path**: Instead of controlling the car's position in the entire plane, we focus on its position along this path.\n",
"> - **One-Dimensional Control**: Adjust the car's speed to control its progress along the path, effectively turning a complex navigation task into controlling how fast it moves forward.\n",
">\n",
"> By simplifying the problem in this way, we can more easily manage the car's movement while still respecting all its physical constraints."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"\n",
"### Partitioning by projection\n",
"\n",
"If you could project our two-dimensional space onto\n",
"a one-dimensional space S, then you could use KITT’s position in S for one-dimensional control. The figure below depicts this idea. The line represents the trajectory that you would want to follow.\n",
"\n",
"Let KITT's trajectory be given by, \n",
"\n",
"$$ \\mathbf{x}(t) = \\left[ x(t), y(t) \\right]^T $$\n",
"\n",
"You can define the projection of \\((x, y)\\) onto \\(S\\) by\n",
"$$\n",
"z = z(x, y)\n",
"$$\n",
"\n",
"where $z \\in S$. The projected position $z$ can be regarded as the distance you travel on $ S $. KITT's speed should then be continuously controlled so that $z$ approaches the desired distance without overshooting. Thus, using $z$ as a measure for distance allows for one-dimensional control in a plane.\n",
"\n",
"This approach requires knowing KITT's trajectory: you need an *estimate* of KITT's future movement. This is given by the planned trajectory. You can now state your wanted projection: The projected position $z$ is given by the arc length of the planned trajectory from KITT's current position to its target position. This allows for velocity control along any trajectory in a plane.\n",
"\n",
" <img src=\"trajectory.jpg\" alt=\"KITTtraject-figure\" width=\"250px\">\n",
"\n",
"*KITT's trajectory*\n",
"<!--\n",
"```{figure} trajectory.jpg\n",
"---\n",
"height/width: 150px\n",
"name: KITTtraject-figure\n",
"---\n",
"KITT's trajectory\n",
"```\n",
"-->\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"vscode": {
"languageId": "plaintext"
}
},
"source": [
"### Steering model\n",
"\n",
"If we steer a car by making the front wheels make an angle to the baseline, then the car will follow a circle. The angle $\\phi$ of the wheels will determine the radius $R$ of the circle (if the angle is zero, then the radius will become infinity: driving straight is a limit case). The dynamic model for driving on a line remains valid, except that it now describes the distance we drive on the circle.\n",
"\n",
"Thus, the hard part at this point is to derive a relation between the angle $\\phi$ and the radius $R$. Consider KITT with distance L between both wheel axes. The rear axis of the car is located at [0; 0]\n",
"and the front axis at [L; 0] in a way that they are parallel to each other. The front wheels are turned an angle $\\phi$ relative to the positive x-axis. If the car drives at a speed $v$, then after a very small time $∆t$, the rear axis\n",
"is located at:\n",
"$$\n",
"\\begin{bmatrix}v\\, \\cos(\\phi)\\Delta t\\\\0 \\end{bmatrix}\n",
"$$\n",
"and the front axis at:\n",
"$$\n",
"\\begin{bmatrix}L + v\\, \\cos(\\phi)\\Delta t\\\\ v\\, \\sin(\\phi)\\Delta t \\end{bmatrix}\n",
"$$\n",
"The situation is sketched below.\n",
"\n",
"\n",
"<img src=\"steering1.png\" alt=\"KITTsteering-figure\" width=\"250px\">\n",
"\n",
"\n",
"*Visualization of KITT steering*\n",
"\n",
"The car was first parallel to the positive x-axis. Therefore, if $\\theta$ denotes the angle of the car relative to the\n",
"positive x-axis,\n",
"$$\n",
"θ(t) = 0\n",
"$$\n",
"and\n",
"$$\n",
"θ(t + ∆t) = \\arctan( \\frac{v\\, \\sin (φ)∆t} {L} )\n",
"$$\n",
".\n",
"We can now evaluate the rate at which KITT turns by\n",
"$$\n",
"\\begin{aligned}\n",
"\\frac{d}{dt}\\theta(t) &= \\lim_{\\Delta t \\to 0} \\frac{\\theta(t + \\Delta t) - \\theta(t)}{\\Delta t}\\\\\n",
"&= \\lim_{\\Delta t \\to 0} \\frac{\\arctan[v\\, \\sin(\\phi) \\Delta t / L]}{\\Delta t}\\\\\n",
"&= \\lim_{\\Delta t \\to 0} (1 + \\frac{v^2\\, \\sin(\\phi) \\Delta t^2]}{L^2})^{-1}\\; \\frac{v\\, \\sin(\\phi)}{L} \\\\\n",
"&= \\frac{v\\, \\sin(\\phi)}{L}\n",
"\\end{aligned}\n",
"$$\n",
"We have found a relationship between the angle of KITT’s wheels and the rate at which KITT turns."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Following your goal\n",
"\n",
"When calculating the accelleration of KITT, you assumed that you were able to let KITT follow any predefined trajectory. You will now design a controller which keeps KITT on track.\n",
"\n",
"Intuitively, you will think of a solution where you know your current position/orientation, and you always steer towards the target. Once you are oriented towards the target, your \"angle error\" is zero, and you just have to drive straight. In all other cases, the angle error determines how much you need to steer. It is easy to see that this approach might work, but also might become unstable once you are very close to the target.\n",
"\n",
"Suppose KITT is driving on its trajectory $S$, where its orientation with respect to the $x$-axis is given by $\\theta$. Let the angle tangent to its trajectory $S$ be given by $\\theta_t$. Ideally, this angle should be equal to KITT's orientation. If KITT's orientation deviates from this angle, KITT should turn its wheels to steer back. This motivates a feedback law given by\n",
"\n",
"$$\n",
"\\phi = -k(\\theta - \\theta_t)\n",
"$$\n",
"\n",
"for a positive $k$. Substituting in the rate at which KITT turns yields the autonomous non-linear system\n",
"\n",
"$$\n",
"L \\,\\frac{d}{dt}\\theta \\;+\\; v\\, \\sin(k(\\theta-\\theta_t)) = 0\\,.\n",
"$$\n",
"\n",
"You should choose $k$ such that this system is asymptotically stable. To investigate the stability, you introduce a potential function (error function) given by\n",
"\n",
"$$\n",
"T(\\theta) = \\frac{1}{2}(\\theta-\\theta_t)^2 \\,.\n",
"$$\n",
"\n",
"A first observation is that $T(\\theta)=0$ if and only if $\\theta=\\theta_t$, which is exactly what you want. Second, notice that $T(\\theta) \\ge 0$. you can conclude that $T$'s minimum corresponds to our equilibrium point. Notice that\n",
"\n",
"$$\n",
"\\frac{d}{dt} T(\\theta) = -(\\theta-\\theta_t)\\frac{v\\, \\sin{(k(\\theta-\\theta_t))}}{L}.\n",
"$$\n",
"\n",
"The figure below depicts both $T(\\theta)$ and its time-derivative.\n",
"\n",
"\n",
"<img src=\"potentialfunction.jpg\" alt=\"KITTpotential-figure\" width=\"250px\">\n",
"\n",
"*Graph of potential function and it's derivative*\n",
"\n",
"Consider KITT's orientation at any instant. If $ |\\theta| < \\theta_m $, then by the figure above the potential function will have a negative derivative. But then it will decrease over time and will keep decreasing until $\\theta = \\theta_t$. So in conclusion, if $ |\\theta| < \\theta_m $, then $\\theta$ will converge to its equilibrium point. By inspection of Equation (\\ref{eq:lyapunov}) you can now state that Equation (\\ref{eq:angle-diffeq}) is locally asymptotically stable for any\n",
"\n",
"$$\n",
"-\\frac{\\pi}{k} < \\theta - \\theta_t < \\frac{\\pi}{k}.\n",
"$$\n",
"\n",
"This treatment is also called the investigation of *Lyapunov stability*, where $T$ is called the *Lyapunov function*. It is an extensive topic in the control literature."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.12.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

51
appendix/Path-planning.ipynb Normal file
View File

@ -0,0 +1,51 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## More advanced methods for path planning \n",
"**⚠️ Optional advanced method**\n",
"The contents below cover advanced topics that not all students are expected to understand.\n",
"\n",
"\n",
"Path planning is essential in robotics and autonomous driving applications. The A* algorithm is a popular pathfinding and graph traversal algorithm that can find the shortest path between two points on a grid. However, standard A* doesn't account for vehicle constraints like turning radius and orientation. To address this, you can use the Hybrid A* algorithm, which considers the vehicle's kinematics, allowing for feasible paths that a real vehicle can follow."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"incase you would like to follow such an approach the folloiwng steps are suggested : \n",
"\n",
"1. Create an Occupancy Map: Represent the environment with possible obstacles and borders.\n",
"\n",
"2. Impose Nonholonomic Constraints: Introduce vehicle kinematic constraints in your path finding algroithm or throw away the paths that dont fit your requirments. \n",
"\n",
"4. plan Path Using Hybrid A*: Find a feasible path that respects the vehicle's movement capabilities."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"So Hybrid A* is the algorithm that extends the A* search by incorporating the vehicle's kinematic model into the path planning process. Instead of simply moving from one grid cell to adjacent cells, Hybrid A* simulates the vehicle's possible movements based on its steering capabilities and motion constraints. This involves generating continuous motion primitives—small, feasible movements like moving forward, turning left, or turning right—which are used to expand the search tree in a way that reflects the vehicle's actual movement abilities.\n",
"\n",
"A good begginer freindly guide or an introduction on it can be found on https://medium.com/@junbs95/gentle-introduction-to-hybrid-a-star-9ce93c0d7869 . The algorithm has been widely adopted and implemented in various platforms, including ROS (Robot Operating System), Unity, Python, and MATLAB, and continues to be a foundational method in the field of robotic motion planning so you might also find existing implmentation of it as well."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.12.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

BIN
appendix/focusrite_back.JPG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 551 KiB

BIN
appendix/focusrite_front.JPG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 590 KiB

126
appendix/obstacle.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

BIN
appendix/potentialfunction.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 52 KiB

218
appendix/state-tracking.ipynb Normal file
View File

File diff suppressed because one or more lines are too long

BIN
appendix/steering1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 78 KiB

BIN
appendix/trajectory.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 33 KiB

BIN
files/Datasheets/ASIO4ALL v2 Instruction Manual.pdf Normal file
View File

Binary file not shown.

BIN
files/Datasheets/KITT datasheet.pdf Normal file
View File

Binary file not shown.

BIN
files/Datasheets/srf02.pdf Normal file
View File

Binary file not shown.

BIN
files/PyAudio-0.2.14-cp312-cp312-win_amd64.whl Normal file
View File

Binary file not shown.

98
files/Recordings/kitt_distance_data_160.csv Normal file
View File

@ -0,0 +1,98 @@
Time,Distance_L,Distance_R
0.00011372566223144531,420.0,420.0
0.10209083557128906,419.9238418107576,419.9238418107576
0.21064186096191406,418.533354225322,418.533354225322
0.31570887565612793,417.02445089103463,417.02445089103463
0.41739773750305176,413.9238229052188,413.9238229052188
0.5211067199707031,410.187054702697,410.187054702697
0.6232409477233887,405.1825755950301,405.1825755950301
0.7249636650085449,399.71168670552174,399.71168670552174
0.8300948143005371,395.4748642615542,395.4748642615542
0.9351727962493896,388.67927709790797,388.67927709790797
1.0367038249969482,381.6479899001781,381.6479899001781
1.138467788696289,374.28748217794805,374.28748217794805
1.2397549152374268,365.9137119461212,365.9137119461212
1.3399817943572998,360.16307146840586,360.16307146840586
1.4448387622833252,351.5179981679535,351.5179981679535
1.5494718551635742,342.31698042086896,342.31698042086896
1.6546688079833984,332.24129171679874,332.24129171679874
1.7558887004852295,322.36988612162276,322.36988612162276
1.861264705657959,315.2824323842395,315.2824323842395
1.9664387702941895,304.65547874979325,304.65547874979325
2.071542739868164,294.1879323336513,294.1879323336513
2.1734578609466553,283.18999593980607,283.18999593980607
2.273740768432617,271.8494469445087,271.8494469445087
2.374274730682373,264.2030968213156,264.2030968213156
2.479445695877075,252.82513425714143,252.82513425714143
2.580343723297119,241.27733858211417,241.27733858211417
2.6854517459869385,229.46883985356294,229.46883985356294
2.790614604949951,221.52177215574147,221.52177215574147
2.895768642425537,209.55264849952232,209.55264849952232
2.9976747035980225,197.18525936030898,197.18525936030898
3.10280179977417,184.99453106482378,184.99453106482378
3.2073137760162354,172.53315134055117,172.53315134055117
3.3123998641967773,164.06996029892434,164.06996029892434
3.4171998500823975,151.3058299464745,151.3058299464745
3.521847724914551,139.20541660353422,139.20541660353422
3.6228177547454834,126.83989941490631,126.83989941490631
3.7252848148345947,118.20880807434042,118.20880807434042
3.82853364944458,105.74174307611281,105.74174307611281
3.931508779525757,93.46623946464007,93.46623946464007
4.033491849899292,80.39986388243835,80.39986388243835
4.140361785888672,67.64203766780656,67.64203766780656
4.245542764663696,58.97649714895937,58.97649714895937
4.346668720245361,46.10398218180427,46.10398218180427
4.448192596435547,32.94237998545219,32.94237998545219
4.553370714187622,24.200132911884907,24.200132911884907
4.657504558563232,12.178423393430137,12.178423393430137
4.762639760971069,1.2866701886258625,1.2866701886258625
4.866015911102295,inf,inf
4.97174072265625,inf,inf
5.076884984970093,inf,inf
5.181800842285156,inf,inf
5.284254789352417,inf,inf
5.3884899616241455,inf,inf
5.494657754898071,inf,inf
5.5953285694122314,inf,inf
5.696522951126099,inf,inf
5.801664590835571,inf,inf
5.9051127433776855,inf,inf
6.010413646697998,inf,inf
6.1141517162323,inf,inf
6.2194297313690186,inf,inf
6.323574781417847,inf,inf
6.42747688293457,inf,inf
6.532923698425293,inf,inf
6.638104677200317,inf,inf
6.743362665176392,inf,inf
6.8450517654418945,inf,inf
6.9493749141693115,inf,inf
7.068382978439331,inf,inf
7.169364929199219,inf,inf
7.271874666213989,inf,inf
7.375161647796631,inf,inf
7.480349779129028,inf,inf
7.5854737758636475,inf,inf
7.688393831253052,inf,inf
7.79346776008606,inf,inf
7.895949602127075,inf,inf
8.001094818115234,inf,inf
8.105218648910522,inf,inf
8.210288763046265,inf,inf
8.312245607376099,inf,inf
8.417306661605835,inf,inf
8.518728971481323,inf,inf
8.622567892074585,inf,inf
8.72764778137207,inf,inf
8.83106780052185,inf,inf
8.932540893554688,inf,inf
9.037699699401855,inf,inf
9.140471696853638,inf,inf
9.245607852935791,inf,inf
9.351115703582764,inf,inf
9.455540895462036,inf,inf
9.560698747634888,inf,inf
9.664724826812744,inf,inf
9.767396688461304,inf,inf
9.873282670974731,inf,inf
9.975827932357788,inf,inf
1 Time Distance_L Distance_R
2 0.00011372566223144531 420.0 420.0
3 0.10209083557128906 419.9238418107576 419.9238418107576
4 0.21064186096191406 418.533354225322 418.533354225322
5 0.31570887565612793 417.02445089103463 417.02445089103463
6 0.41739773750305176 413.9238229052188 413.9238229052188
7 0.5211067199707031 410.187054702697 410.187054702697
8 0.6232409477233887 405.1825755950301 405.1825755950301
9 0.7249636650085449 399.71168670552174 399.71168670552174
10 0.8300948143005371 395.4748642615542 395.4748642615542
11 0.9351727962493896 388.67927709790797 388.67927709790797
12 1.0367038249969482 381.6479899001781 381.6479899001781
13 1.138467788696289 374.28748217794805 374.28748217794805
14 1.2397549152374268 365.9137119461212 365.9137119461212
15 1.3399817943572998 360.16307146840586 360.16307146840586
16 1.4448387622833252 351.5179981679535 351.5179981679535
17 1.5494718551635742 342.31698042086896 342.31698042086896
18 1.6546688079833984 332.24129171679874 332.24129171679874
19 1.7558887004852295 322.36988612162276 322.36988612162276
20 1.861264705657959 315.2824323842395 315.2824323842395
21 1.9664387702941895 304.65547874979325 304.65547874979325
22 2.071542739868164 294.1879323336513 294.1879323336513
23 2.1734578609466553 283.18999593980607 283.18999593980607
24 2.273740768432617 271.8494469445087 271.8494469445087
25 2.374274730682373 264.2030968213156 264.2030968213156
26 2.479445695877075 252.82513425714143 252.82513425714143
27 2.580343723297119 241.27733858211417 241.27733858211417
28 2.6854517459869385 229.46883985356294 229.46883985356294
29 2.790614604949951 221.52177215574147 221.52177215574147
30 2.895768642425537 209.55264849952232 209.55264849952232
31 2.9976747035980225 197.18525936030898 197.18525936030898
32 3.10280179977417 184.99453106482378 184.99453106482378
33 3.2073137760162354 172.53315134055117 172.53315134055117
34 3.3123998641967773 164.06996029892434 164.06996029892434
35 3.4171998500823975 151.3058299464745 151.3058299464745
36 3.521847724914551 139.20541660353422 139.20541660353422
37 3.6228177547454834 126.83989941490631 126.83989941490631
38 3.7252848148345947 118.20880807434042 118.20880807434042
39 3.82853364944458 105.74174307611281 105.74174307611281
40 3.931508779525757 93.46623946464007 93.46623946464007
41 4.033491849899292 80.39986388243835 80.39986388243835
42 4.140361785888672 67.64203766780656 67.64203766780656
43 4.245542764663696 58.97649714895937 58.97649714895937
44 4.346668720245361 46.10398218180427 46.10398218180427
45 4.448192596435547 32.94237998545219 32.94237998545219
46 4.553370714187622 24.200132911884907 24.200132911884907
47 4.657504558563232 12.178423393430137 12.178423393430137
48 4.762639760971069 1.2866701886258625 1.2866701886258625
49 4.866015911102295 inf inf
50 4.97174072265625 inf inf
51 5.076884984970093 inf inf
52 5.181800842285156 inf inf
53 5.284254789352417 inf inf
54 5.3884899616241455 inf inf
55 5.494657754898071 inf inf
56 5.5953285694122314 inf inf
57 5.696522951126099 inf inf
58 5.801664590835571 inf inf
59 5.9051127433776855 inf inf
60 6.010413646697998 inf inf
61 6.1141517162323 inf inf
62 6.2194297313690186 inf inf
63 6.323574781417847 inf inf
64 6.42747688293457 inf inf
65 6.532923698425293 inf inf
66 6.638104677200317 inf inf
67 6.743362665176392 inf inf
68 6.8450517654418945 inf inf
69 6.9493749141693115 inf inf
70 7.068382978439331 inf inf
71 7.169364929199219 inf inf
72 7.271874666213989 inf inf
73 7.375161647796631 inf inf
74 7.480349779129028 inf inf
75 7.5854737758636475 inf inf
76 7.688393831253052 inf inf
77 7.79346776008606 inf inf
78 7.895949602127075 inf inf
79 8.001094818115234 inf inf
80 8.105218648910522 inf inf
81 8.210288763046265 inf inf
82 8.312245607376099 inf inf
83 8.417306661605835 inf inf
84 8.518728971481323 inf inf
85 8.622567892074585 inf inf
86 8.72764778137207 inf inf
87 8.83106780052185 inf inf
88 8.932540893554688 inf inf
89 9.037699699401855 inf inf
90 9.140471696853638 inf inf
91 9.245607852935791 inf inf
92 9.351115703582764 inf inf
93 9.455540895462036 inf inf
94 9.560698747634888 inf inf
95 9.664724826812744 inf inf
96 9.767396688461304 inf inf
97 9.873282670974731 inf inf
98 9.975827932357788 inf inf

BIN
files/Student Recordings/record_x109_y76.wav Normal file
View File

Binary file not shown.

BIN
files/Student Recordings/record_x143_y296.wav Normal file
View File

Binary file not shown.

BIN
files/Student Recordings/record_x150_y185.wav Normal file
View File

Binary file not shown.

BIN
files/Student Recordings/record_x178_y439.wav Normal file
View File

Binary file not shown.

BIN
files/Student Recordings/record_x232_y275.wav Normal file
View File

Binary file not shown.

BIN
files/Student Recordings/record_x4_y_hidden_1.wav Normal file
View File

Binary file not shown.

BIN
files/Student Recordings/record_x64_y40.wav Normal file
View File

Binary file not shown.

BIN
files/Student Recordings/record_x82_y399.wav Normal file
View File

Binary file not shown.

BIN
files/Student Recordings/record_x_y_hidden_2.wav Normal file
View File

Binary file not shown.

BIN
files/Student Recordings/record_x_y_hidden_3.wav Normal file
View File

Binary file not shown.

BIN
files/Student Recordings/reference.wav Normal file
View File

Binary file not shown.

BIN
pictures/Ackermann-steering-geometry-for-the-front-steer-angles-of-the-inner-and-outer-wheels.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

BIN
pictures/Beacon.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 39 KiB

BIN
pictures/FinalChallenge.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

BIN
pictures/KITT_leds.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 932 KiB

BIN
pictures/KITTwind.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 124 KiB

BIN
pictures/Step1.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 10 KiB

BIN
pictures/Step2.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 90 KiB

BIN
pictures/Step3.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 56 KiB

BIN
pictures/Step4.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 110 KiB

BIN
pictures/Step5.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 38 KiB

BIN
pictures/Step6.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 298 KiB

BIN
pictures/Step6Arrow.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 26 KiB

BIN
pictures/Step7.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 10 KiB

BIN
pictures/TDOA.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 90 KiB

BIN
pictures/XFplot.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

BIN
pictures/axisdef.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.6 KiB

BIN
pictures/bicycle.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 192 KiB

BIN
pictures/communication_overview.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 105 KiB

BIN
pictures/figaudiocode.pdf Normal file
View File

Binary file not shown.

BIN
pictures/figurejoin.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 665 KiB

BIN
pictures/gui_example.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 897 KiB

BIN
pictures/header.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 116 KiB

BIN
pictures/intro_setup.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 160 KiB

BIN
pictures/logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.6 KiB

BIN
pictures/mcu_board.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.4 MiB

BIN
pictures/potential.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 112 KiB

BIN
pictures/potentialfunction.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

BIN
pictures/projectBD.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 106 KiB

BIN
pictures/projection.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 71 KiB

BIN
pictures/scrum_backlog-1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 14 KiB

BIN
pictures/scrum_chart-1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 156 KiB

BIN
pictures/scrum_overview-1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 165 KiB

BIN
pictures/srf02-ultrasonic-sensor.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 15 KiB

BIN
pictures/steering.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 54 KiB

BIN
pictures/steps.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 234 KiB

BIN
pictures/systemoverview.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

BIN
pictures/systemoverview1.pdf Normal file
View File

Binary file not shown.

1
pictures/test_pic Normal file
View File

@ -0,0 +1 @@

BIN
pictures/trajectory.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 30 KiB

BIN
pictures/traxxas_e-maxx.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 244 KiB

BIN
pictures/truck.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 47 KiB

11
requirements.txt Normal file
View File

@ -0,0 +1,11 @@
jupyter-book
matplotlib
numpy
pyserial
scipy
statsmodels
ipywidgets
ipycanvas
shapely
sounddevice
tk

1
student-code Submodule

@ -0,0 +1 @@
Subproject commit eb982ad8ab46ed773d72e1e79597b23fc5e2ebba