SCHOOL

School of Engineering

ACADEMIC UNIT

Dept. of Electrical & Computer Engineering

LEVEL OF STUDIES

Undergraduate

COURSE CODE

ECE_AK701

SEMESTER

7^th

COURSE TITLE

Information Theory

INDEPENDENT TEACHING ACTIVITIES
if credits are awarded for separate components of the course, e.g. lectures, laboratory exercises, etc. If the credits are awarded for the whole of the course, give the weekly teaching hours and the total credits

WEEKLY TEACHING HOURS

CREDITS

Lectures

3

5

Tutorials

1

Add rows if necessary. The organisation of teaching and the teaching methods used are described in detail at (d).

4

5

COURSE TYPE

general background,
special background, specialised general knowledge, skills development

The course builds specialized background for the engineer in the fields of Telecommunications and Information Technology. It is a mandatory course.

PREREQUISITE COURSES:

Probability & Statistics, Linear Algebra

LANGUAGE OF INSTRUCTION and EXAMINATIONS:

Greek

IS THE COURSE OFFERED TO ERASMUS STUDENTS

No

COURSE WEBSITE (URL)

Learning outcomes

The course learning outcomes, specific knowledge, skills and competences of an appropriate level, which the students will acquire with the successful completion of the course are described.

Consult Appendix A

·    Description of the level of learning outcomes for each qualifications cycle, according to the Qualifications Framework of the European Higher Education Area

·    Descriptors for Levels 6, 7 & 8 of the European Qualifications Framework for Lifelong Learning and Appendix B

·    Guidelines for writing Learning Outcomes

Information theory is the science of operations on data such as compression, storage, and communication. It is among the few disciplines fortunate to have a precise date of birth: 1948, with the publication of Claude E. Shannon's paper entitled “A Mathematical Theory of Communication”.

Our course will explore the basic concepts of Information theory. It is a prerequisite for research in this area, and highly recommended for students planning to delve into the fields of communications, data compression, and statistical signal processing. The intimate acquaintance that we will gain with measures of information and uncertainty - such as mutual information, entropy, and relative entropy - would be invaluable also for students, researchers, and practitioners in fields ranging from neuroscience to machine learning. Students also will gain an appreciation for the interplay between information theory, combinatorics, probability, and statistics.

Furthermore, students will also gain deep knowledge into the code theory applicable to various fields of exchanging end safeguarding information.

At the end of the course the student will be in a position

• To model and measure information in various contexts, primarily in telecommunications
• The thresholds of information contexts (entropy)
• Algorithms for coding both of sources and of communication channels.
• Deep knowledge of error coding techniques and their performance
• Apply his/her knowledge into various fields of technology and sectors of daily life

General Competences

Taking into consideration the general competences that the degree-holder must acquire (as these appear in the Diploma Supplement and appear below), at which of the following does the course aim?

Search for, analysis and synthesis of data and information, with the use of the necessary technology

Adapting to new situations

Decision-making

Working independently

Team work

Working in an international environment

Working in an interdisciplinary environment

Production of new research ideas

Project planning and management

Respect for difference and multiculturalism

Respect for the natural environment

Showing social, professional and ethical responsibility and sensitivity to gender issues

Criticism and self-criticism

Production of free, creative and inductive thinking

……

Others…

…….

• Search for, analysis and synthesis of data and information, with the use of the necessary technology
• Adapting to new situations
• Decision-making
• Working independently
• Working in an international environment
• Working in an interdisciplinary environment
• Production of new research ideas
• Production of free, creative and inductive thinking

This course will cover the following topics:

·        Introduction to Information Theory. What is information, how is it measured?

·        Probability, Entropy, and Inference

·        The Source Coding Theorem, Discrete and memoryless sources of symbols

·        Symbol Codes (Fixed and Variable Length)

·        Stream Codes

·        Communication over a Noisy Channel

·        The Noisy-Channel Coding Theorem

·        Coding and Error Correction Theory.

·        Introduction to Coding Theory. Error Detection. Error Correction.

·        Linear Codes: Generator and Parity Check Matrix. Decoding with Cosets. Decoding with Syndromes. Hamming codes.  Dual Codes.  Perfect Codes.

·        Cyclic Codes: Basic Theory , Encoding and Decoding of Cyclic Codes.

·        Finite Fields (Galouis Fields). BCH Codes: Basic Theory, Encoding and Decoding of BCH Codes with error correction capability of 2 errors or more.

·        Reference to Convolutional Codes: Basic Theory, Encoding and Decoding (Viterbi) of Convolutional Codes

DELIVERY
Face-to-face, Distance learning, etc.

Face to face lectures (4 hours per week [3 hours lectures 1 hour tutorial, PowerPoint presentation. The students are given exercises handouts for homework.

USE OF INFORMATION AND COMMUNICATIONS TECHNOLOGY
Use of ICT in teaching, laboratory education, communication with students

Use of eclass for exchange of documents and communication.

TEACHING METHODS

The manner and methods of teaching are described in detail.

Lectures, seminars, laboratory practice, fieldwork, study and analysis of bibliography, tutorials, placements, clinical practice, art workshop, interactive teaching, educational visits, project, essay writing, artistic creativity, etc.

The student's study hours for each learning activity are given as well as the hours of non-directed study according to the principles of the ECTS

STUDENT PERFORMANCE EVALUATION

Description of the evaluation procedure

Language of evaluation, methods of evaluation, summative or conclusive, multiple choice questionnaires, short-answer questions, open-ended questions, problem solving, written work, essay/report, oral examination, public presentation, laboratory work, clinical examination of patient, art interpretation, other

Specifically-defined evaluation criteria are given, and if and where they are accessible to students.

Assessment includes mid-term exams (optional) and final term exams

The final grade is calculated by the formula

Final Grade = Max [ (Mid-term exam x 50% + Final Exam x 50%), (Final Exam) ]

- Suggested bibliography:

1. Information Theory, THOMAS M. COVER - JOY A. THOMAS

2. Information Theory, Inference, and Learning Algorithms, David McKay

3. Information Theory and Reliable Communication, Robert G. Gallager

4. Notes and Slide Presentations

- Related academic journals: