Department of Electrical and Electronics Engineering

ACADEMICS
Course Details

ELE737 - Fundamentals of Information Theory

2024-2025 Fall term information
The course is open this term
Supervisor(s)
Name Surname Position Section
Dr. Emre Aktaş Supervisor
Weekly Schedule by Sections
Section Day, Hours, Place
All sections Tuesday, 13:40 - 16:30, SS

Timing data are obtained using weekly schedule program tables. To make sure whether the course is cancelled or time-shifted for a specific week one should consult the supervisor and/or follow the announcements.

ELE737 - Fundamentals of Information Theory
Program Theoretıcal hours Practical hours Local credit ECTS credit
PhD 3 0 3 10
Obligation : Elective
Prerequisite courses : -
Concurrent courses : -
Delivery modes : Face-to-Face
Learning and teaching strategies : Lecture, Question and Answer, Problem Solving
Course objective : The objective of the course is to introduce ? the notion of entropy and information ? the fundamental limits of data compression ? the fundamental limits of data transmission systems.
Learning outcomes : Learn and use the main mathematical tools of information theory that quantify and relate information Learn fundamental limits for systems that store and compress data Learn fundamental methods of source coding Learn fundamental limits for systems that communicate data Utilize information theory in order to gain insight of and design any system that stores, processes, or communicates information
Course content : Introduction, review of probability, Entropy, relative entropy, mutual information, inequalities, The asymptotic equipartition property, Data compression, Channel capacity, Differential entropy, the Gaussian channel, Network information theory.
References : Elements of Information Theory, Cover and Thomas, Wiley Interscience; Gallager, "Claude E. Shannon: A Retrospective on His Life, Work, and Impact", IEEE; Trans. Inform. Theory, vol.47, no.7, Nov. 2001; Wyner, "Fundamental Limits in Information Theory", Proc. of the IEEE, vol.69, no.2,; Feb. 1981; Verdu, "Fifty Years of Shannon Theory", IEEE Trans. Inform. Theory, vol.44, no.6,; Oct. 1998
Course Outline Weekly
Weeks Topics
1 Review of probability theory, entropy
2 Relative entropy and mutual information
3 Jensen?s inequality and its consequences
4 Asymptotic equipartition property
5 Data compression and Kraft inequality
6 Optimal codes, Huffman codes
7 Shannon-Fano-Elias coding
8 Midterm Exam
9 Channel capacity examples
10 Channel coding theorem
11 Fano?s inequality and the converse to the coding theorem
12 Differential entropy
13 Gaussian channel
14 Network information theory
15 Final exam
16 Final exam
Assessment Methods
Course activities Number Percentage
Attendance 0 0
Laboratory 0 0
Application 0 0
Field activities 0 0
Specific practical training 0 0
Assignments 1 5
Presentation 0 0
Project 0 0
Seminar 1 5
Quiz 0 0
Midterms 1 40
Final exam 0 50
Total 100
Percentage of semester activities contributing grade success 50
Percentage of final exam contributing grade success 50
Total 100
Workload and ECTS Calculation
Course activities Number Duration (hours) Total workload
Course Duration 14 3 42
Laboratory 0 0 0
Application 0 0 0
Specific practical training 0 0 0
Field activities 0 0 0
Study Hours Out of Class (Preliminary work, reinforcement, etc.) 14 12 168
Presentation / Seminar Preparation 0 0 0
Project 1 25 25
Homework assignment 1 5 5
Quiz 0 0 0
Midterms (Study duration) 0 0 0
Final Exam (Study duration) 1 25 25
Total workload 31 70 265
Matrix Of The Course Learning Outcomes Versus Program Outcomes
Key learning outcomes Contribution level
1 2 3 4 5
1. Has highest level of knowledge in certain areas of Electrical and Electronics Engineering.
2. Has knowledge, skills and and competence to develop novel approaches in science and technology.
3. Follows the scientific literature, and the developments in his/her field, critically analyze, synthesize, interpret and apply them effectively in his/her research.
4. Can independently carry out all stages of a novel research project.
5. Designs, plans and manages novel research projects; can lead multidisiplinary projects.
6. Contributes to the science and technology literature.
7. Can present his/her ideas and works in written and oral forms effectively; in Turkish or English.
8. Is aware of his/her social responsibilities, evaluates scientific and technological developments with impartiality and ethical responsibility and disseminates them.
1: Lowest, 2: Low, 3: Average, 4: High, 5: Highest
General Information | Course & Exam Schedules | Real-time Course & Classroom Status
Undergraduate Curriculum | Minor Program For Non-departmental Students | Open Courses, Sections and Supervisors | Weekly Course Schedule | Examination Schedules | Information for Registration | Prerequisite and Concurrent Courses | Legal Info and Documents for Internship | Academic Advisors for Undergraduate Program | Information for ELE 401-402 Graduation Project | Virtual Exhibitions of Graduation Projects | Erasmus+ Program | Program Educational Objectives & Student Outcomes | ECTS Course Catalog | HU Registrar's Office
Graduate Curriculum | Open Courses and Supervisors | Weekly Course Schedule | Final Examinations Schedule | Schedule of Graduate Thesis Defences and Seminars | Information for Registration | ECTS Course Catalog - Master's Degree | ECTS Course Catalog - PhD Degree | HU Graduate School of Science and Engineering