Neural Networks - Syllabus

Course Overview and Structure

Embark on a profound academic exploration as you delve into the Neural Networks course () within the distinguished Tribhuvan university's CSIT department. Aligned with the 2074 Syllabus, this course (CSC372) seamlessly merges theoretical frameworks with practical sessions, ensuring a comprehensive understanding of the subject. Rigorous assessment based on a 60 + 20 + 20 marks system, coupled with a challenging passing threshold of , propels students to strive for excellence, fostering a deeper grasp of the course content.

This 3 credit-hour journey unfolds as a holistic learning experience, bridging theory and application. Beyond theoretical comprehension, students actively engage in practical sessions, acquiring valuable skills for real-world scenarios. Immerse yourself in this well-structured course, where each element, from the course description to interactive sessions, is meticulously crafted to shape a well-rounded and insightful academic experience.


Course Description:

The course introduces the underlying principles and design of Neural Network. The course covers the basics concepts of Neural Network including: its architecture, learning processes, single layer and multilayer perceptron followed by Recurrent Neural Network

Course Objective:

The course objective is to demonstrate the concept of supervised learning, unsupervised learning in conjunction with different architectures of Neural Network

Units

Key Topics

  • Introduction to E-commerce
    IN-1

    Overview of E-commerce and its significance in the digital age.

  • E-business vs E-commerce
    IN-2

    Understanding the differences between E-business and E-commerce.

  • Features of E-commerce
    IN-3

    Key characteristics and benefits of E-commerce.

  • Pure vs Partial E-commerce
    IN-4

    Types of E-commerce models and their applications.

  • History of E-commerce
    IN-5

    Evolution and development of E-commerce over time.

  • E-commerce Framework
    IN-6

    Understanding the components of E-commerce framework including People, Public Policy, Marketing and Advertisement, Support Services, and Business Partnerships.

  • Types of E-commerce
    IN-7

    Overview of different types of E-commerce including B2C, B2B, C2B, C2C, M-Commerce, U-commerce, Social-Ecommerce, and Local E-commerce.

  • Challenges in E-commerce
    IN-8

    Common obstacles and difficulties faced in E-commerce.

Key Topics

  • Moment of Inertia and Torque
    RO-1

    This topic covers the concept of moment of inertia and torque, including their definitions, units, and applications in rotational dynamics.

  • Rotational Kinetic Energy
    RO-2

    This topic explores the concept of rotational kinetic energy, including its definition, formula, and relationship with rotational motion.

  • Conservation of Angular Momentum
    RO-3

    This topic discusses the principle of conservation of angular momentum, including its definition, examples, and applications in rotational dynamics.

  • Oscillation of a Spring
    RO-4

    This topic covers the oscillatory motion of a spring, including the concepts of frequency, period, amplitude, phase angle, and energy.

  • The Batch Perceptron Algorithm
    RO-5

    A description of the Batch Perceptron Algorithm, a variant of the Perceptron algorithm that processes multiple inputs in batches. This topic covers the implementation details of the Batch Perceptron Algorithm.

Key Topics

  • Introduction to E-Governance Models
    MO-1

    Overview of E-Governance models and their significance in digital governance.

  • Broadcasting / Wider Dissemination Model
    MO-2

    A model of E-Governance that focuses on disseminating information to citizens through various channels.

  • Critical Flow Model
    MO-3

    A model that emphasizes the critical flow of information and services between government and citizens.

  • Comparative Analysis Model
    MO-4

    A model that involves comparative analysis of different E-Governance initiatives and their outcomes.

  • Mobilization and Lobbying Model
    MO-5

    A model that focuses on mobilizing citizens and lobbying for their rights through E-Governance initiatives.

  • Interactive – Service Model / Government-to-Citizen-to-Government Model (G2C2G)
    MO-6

    A model that enables interactive services between government, citizens, and other stakeholders.

  • Evolution in E-Governance and Maturity Models
    MO-7

    The evolution of E-Governance and the concept of maturity models in achieving good governance.

  • Five Maturity Levels
    MO-8

    The five stages of maturity in E-Governance, from initial to advanced levels.

Key Topics

  • Relational Model Concepts
    TH-1

    This topic covers the fundamental concepts of the relational model, including domains, attributes, tuples, and relations, as well as the characteristics of relations.

  • Relational Model Constraints
    TH-2

    This topic explores the different types of constraints in the relational model, including domain constraints, key constraints, and constraints on null values.

  • Relational Database Schemas
    TH-3

    This topic discusses the concept of relational database schemas, including relational database state, entity integrity, referential integrity, and foreign keys.

  • Update Operations and Transactions
    TH-4

    This topic covers update operations, transactions, and how to deal with constraint violations, including insert, delete, and update operations, as well as restrict, cascade, set null, and set default.

  • Basic Relational Algebra Operations
    TH-5

    This topic introduces basic relational algebra operations, including unary operations (select, project, rename) and binary operations (set theory, Cartesian product, join, and outer join).

  • XML Schema
    TH-6

    Defining the structure and constraints of XML documents using XML Schema.

  • Simple and Complex Types
    TH-7

    Understanding simple and complex data types in XML Schema.

  • XSD Attributes
    TH-8

    Using attributes in XML Schema to provide additional information.

  • Default and Fixed Values
    TH-9

    Specifying default and fixed values for elements and attributes in XML Schema.

  • Facets
    TH-10

    Restricting data types using facets in XML Schema.

  • Patterns and Order Indicators
    TH-11

    Using patterns and order indicators (all, choice, sequence) to define element relationships.

Key Topics

  • Multiple Correlation
    MU-1

    Introduction to multiple correlation, its concept, and application in statistics. Exploring the relationship between multiple variables.

  • Partial Correlation
    MU-2

    Understanding partial correlation, its concept, and application in statistics. Analyzing the relationship between two variables while controlling for other variables.

  • Introduction to Multiple Linear Regression
    MU-3

    Basic concepts and principles of multiple linear regression, including model formulation and estimation. Understanding the relationship between multiple independent variables and a dependent variable.

  • Hypothesis Testing of Multiple Regression
    MU-4

    Testing hypotheses in multiple regression, including significance testing and confidence intervals. Evaluating the overall fit and significance of the regression model.

  • Test of Significance of Regression
    MU-5

    Testing the overall significance of the regression model, including F-test and p-value interpretation. Determining whether the regression model is a good fit to the data.

  • Test of Individual Regression Coefficient
    MU-6

    Testing the significance of individual regression coefficients, including t-test and p-value interpretation. Evaluating the contribution of each independent variable to the regression model.

  • Model Adequacy Tests
    MU-7

    Evaluating the goodness of fit and adequacy of the multiple regression model, including residual analysis and diagnostic plots. Identifying potential issues and limitations of the model.

  • Databases in the Cloud
    MU-8

    Exploring the concept of databases in the cloud and their relevance in modern database administration. This topic covers the benefits and challenges of deploying databases in the cloud.

  • Generalization and Approximation
    MU-9

    Discussion of generalization and approximation capabilities of multilayer perceptrons.

  • Cross Validation and Model Evaluation
    MU-10

    Importance of cross-validation and other techniques for evaluating and selecting neural network models.

  • Complexity Regularization and Network Pruning
    MU-11

    Regularization techniques for controlling model complexity, including network pruning.

  • Virtues and Limitations of Back-Propagation
    MU-12

    Critical evaluation of the strengths and weaknesses of the back-propagation algorithm.

  • Supervised Learning as Optimization
    MU-13

    Formulation of supervised learning as an optimization problem, with implications for neural network training.

  • Convolutional Networks
    MU-14

    Introduction to convolutional neural networks, a specialized type of multilayer perceptron.

  • Nonlinear Filtering
    MU-15

    Applications of multilayer perceptrons to nonlinear filtering and signal processing.

  • Small-Scale vs Large-Scale Learning
    MU-16

    Comparison of small-scale and large-scale learning problems, with implications for neural network design.

Key Topics

  • Introduction to Kernel Methods
    KE-01

    An overview of kernel methods and their significance in neural networks. This topic sets the stage for the rest of the unit.

  • Cover's Theorem on Pattern Separability
    KE-02

    A fundamental theorem in neural networks that establishes the conditions for separability of patterns. This theorem has important implications for neural network design.

  • The Interpolation Problem
    KE-03

    A critical problem in neural networks where the goal is to find a function that passes through a set of given points. This problem is essential in understanding radial-basis function networks.

  • Radial-Basis Function Networks
    KE-04

    A type of neural network that uses radial basis functions as activation functions. This topic explores the architecture and training of RBF networks.

  • K-Means Clustering
    KE-05

    An unsupervised learning algorithm used for clustering data. This topic discusses the application of K-Means clustering in RBF networks.

  • Recursive Least-Squares Estimation of Weight Vectors
    KE-06

    A method for estimating the weight vectors in RBF networks using recursive least-squares estimation. This topic covers the mathematical formulation and implementation of this method.

  • Hybrid Learning Procedure for RBF Networks
    KE-07

    A learning procedure that combines different techniques to train RBF networks. This topic explores the advantages and limitations of this hybrid approach.

  • Kernel Regression and Its Relation to RBF Networks
    KE-08

    A regression technique that uses kernel functions to estimate the target function. This topic discusses the connection between kernel regression and RBF networks.

Key Topics

  • Challenges and Approach of E-government Security
    SE-1

    This topic covers the challenges faced by e-government in terms of security and the approaches to address them. It explores the importance of security in e-government and the ways to mitigate risks.

  • Security Management Model
    SE-2

    This topic introduces a security management model for e-government, outlining the key components and processes involved in ensuring the security of e-government systems.

  • E-Government Security Architecture
    SE-3

    This topic delves into the architecture of e-government security, including the design and implementation of secure systems and infrastructure for e-government services.

  • Security Standards
    SE-4

    This topic covers the security standards and guidelines for e-government, including international standards and best practices for ensuring the security of e-government systems and data.

  • Data Transaction Security
    SE-5

    Security measures for protecting data during transactions in e-commerce.

  • Security Mechanisms
    SE-6

    Various security mechanisms used in e-commerce including cryptography, hash functions, digital signatures, authentication, access controls, intrusion detection systems, and secured socket layer (SSL).

  • javax.servlet.http Package
    SE-7

    Exploring the javax.servlet.http package, including key classes and interfaces. Understanding how to use the package to develop HTTP-based servlets.

  • Handling HTTP Requests and Responses
    SE-8

    Understanding how to handle HTTP requests and responses using servlets, including request and response objects.

Key Topics

  • Dynamic Programming vs Memoization
    DY-10

    Comparison of dynamic programming and memoization, highlighting their similarities and differences.

  • Introduction to Dynamic Driven Recurrent Networks
    DY-01

    Overview of dynamic driven recurrent networks, including their architecture and applications.

  • Recurrent Network Architectures
    DY-02

    Different types of recurrent network architectures, including simple recurrent networks, long short-term memory (LSTM) networks, and gated recurrent units (GRU).

  • Universal Approximation Theorem
    DY-03

    The universal approximation theorem, which states that a recurrent neural network can approximate any continuous function to arbitrary accuracy.

  • Controllability and Observability
    DY-04

    The concepts of controllability and observability in recurrent networks, including their importance in training and stability.

  • Computational Power of Recurrent Networks
    DY-05

    The computational power of recurrent networks, including their ability to learn and represent complex patterns.

  • Learning Algorithms for Recurrent Networks
    DY-06

    Overview of learning algorithms for recurrent networks, including backpropagation through time and real-time recurrent learning.

  • Back Propagation through Time
    DY-07

    The backpropagation through time algorithm, which is used to train recurrent networks.

  • Real-Time Recurrent Learning
    DY-08

    The real-time recurrent learning algorithm, which is used to train recurrent networks in real-time.

  • Vanishing Gradients in Recurrent Networks
    DY-09

    The problem of vanishing gradients in recurrent networks, including its causes and solutions.

  • Adaptivity Considerations
    DY-11

    Considerations for adaptivity in recurrent networks, including the importance of adapting to changing inputs and environments.

  • Case Study: Model Reference Applied to Neurocontrol
    DY-12

    A case study on applying model reference to neurocontrol using recurrent networks.

Lab works

Laboratory works:

Practical should be focused on Single Layer Perceptron, Multilayer Perceptron, Supervised Learning, Unsupervised Learning, Recurrent Neural Network, Linear Prediction and Pattern Classification