[2021-01-26] GraphACT: Accelerating GCN Training on CPU-FPGA Heterogeneous Platforms

1 minute read

Published: January 26, 2021

This paper is a CPU-FPGA heterogenrous platform for GCN training. CPU will do the communication intensive operations, and leave the computation intensive parts to CPU.

Background and Motivation

It is challenging to accelerate Graph Convolutional Networks because: (1) substantial and irregular data communication to propagate information within the graph (2) intensive computation to propagate information along the neural network layers (3) Degree-imbalance ofgraph nodes can significantly degrade the performance of feature propagation.

[2020-07-26]High-Performance Deep-Learning Coprocessor Integrated into x86 SoC with Server-Class CPUs

2 minute read

Published: July 26, 2020

This paper is an industry-track paper in ASPLOS 2020. I used to read more acedemic paper than industry paper, so I lack the knowledge what the focus of the industry. This paper decribes Centaur Technology’s Ncore, the industry’s first high-performance DL coprocessor technology in-target into an x86 SoC with server class CPUs.

[2021-02-20] Analyzing and Mitigating Data Stalls in DNN Training

3 minute read

Published: February 20, 2021

Most DNN acclerator papers I read focus on DNN inference rather than training. From this paper, I learned that the bottleneck for DNN training is I/O for fetching data and CPU side for preprocessing.

Background

[2021-01-26] GraphACT: Accelerating GCN Training on CPU-FPGA Heterogeneous Platforms

1 minute read

Published: January 26, 2021

This paper is a CPU-FPGA heterogenrous platform for GCN training. CPU will do the communication intensive operations, and leave the computation intensive parts to CPU.

Background and Motivation

It is challenging to accelerate Graph Convolutional Networks because: (1) substantial and irregular data communication to propagate information within the graph (2) intensive computation to propagate information along the neural network layers (3) Degree-imbalance ofgraph nodes can significantly degrade the performance of feature propagation.

[2021-01-18] [CS224W] Graph Neural Network

less than 1 minute read

Published: January 18, 2021

Basics of Graph Neural Network

[2021-02-20] Analyzing and Mitigating Data Stalls in DNN Training

3 minute read

Published: February 20, 2021

Most DNN acclerator papers I read focus on DNN inference rather than training. From this paper, I learned that the bottleneck for DNN training is I/O for fetching data and CPU side for preprocessing.

Background

[2021-01-07] Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training

2 minute read

Published: January 07, 2021

This is a paper about GPU memory footprint reduction using selective recomputation during LSTM training process. Although I do not know much about LSTM network, this paper is quite clear and easy to understand.

[2021-01-07] Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training

2 minute read

Published: January 07, 2021

This is a paper about GPU memory footprint reduction using selective recomputation during LSTM training process. Although I do not know much about LSTM network, this paper is quite clear and easy to understand.

[2020-11-14] High-Performance Deep-Learning Coprocessor Integrated into x86 SoC with Server-Class CPUs

3 minute read

Published: November 14, 2020

This is a paper using processor-in-memory(PIM) for NN acceleration in ASPLOS 2020. I know little about PIM techinques. And I don’t know much about the SRAM structure. I learned some basic knowledge of SRAM, PIM from this paper. And I learned a lot of the implementation tools and benchmark measuring tools.

[2021-10-12] ISCA 2021 Section 3A: Machine Learning 1

1 minute read

Published: October 12, 2021

This reading blog is about three papers in Section 3A: Machine Learning 1 of ISCA 2021.

RaPiD: AI Accelerator for Ultra-Low Precision Training and Inference

This accelerator supports mixed precisions for both training and inference: 16 and 8-bit floating-point and 4 and 2-bit fixed point. It imporves both performance(TOPS) and energy efficiency(TOPS/W) at ultra-low preceision. In my opinion, this work contributes more on the engineering part (architecture).

[2021-09-24] Ten Lessons From Three Generations Shaped Google’s TPUv4i : Industrial Product

2 minute read

Published: September 24, 2021

I was quite busy during this year’s ISCA, so I did not have the chance to read the papers or watch the videos carefully. I will try to read the ISCA 2021 papers that I am interested in, hopefull before MICRO 2021 starts.

[2020-11-14] High-Performance Deep-Learning Coprocessor Integrated into x86 SoC with Server-Class CPUs

3 minute read

Published: November 14, 2020

This is a paper using processor-in-memory(PIM) for NN acceleration in ASPLOS 2020. I know little about PIM techinques. And I don’t know much about the SRAM structure. I learned some basic knowledge of SRAM, PIM from this paper. And I learned a lot of the implementation tools and benchmark measuring tools.

[2020-11-14] High-Performance Deep-Learning Coprocessor Integrated into x86 SoC with Server-Class CPUs

3 minute read

Published: November 14, 2020

This is a paper using processor-in-memory(PIM) for NN acceleration in ASPLOS 2020. I know little about PIM techinques. And I don’t know much about the SRAM structure. I learned some basic knowledge of SRAM, PIM from this paper. And I learned a lot of the implementation tools and benchmark measuring tools.

[2021-10-12] ISCA 2021 Section 3A: Machine Learning 1

1 minute read

Published: October 12, 2021

This reading blog is about three papers in Section 3A: Machine Learning 1 of ISCA 2021.

RaPiD: AI Accelerator for Ultra-Low Precision Training and Inference

This accelerator supports mixed precisions for both training and inference: 16 and 8-bit floating-point and 4 and 2-bit fixed point. It imporves both performance(TOPS) and energy efficiency(TOPS/W) at ultra-low preceision. In my opinion, this work contributes more on the engineering part (architecture).

[2021-09-24] Ten Lessons From Three Generations Shaped Google’s TPUv4i : Industrial Product

2 minute read

Published: September 24, 2021

I was quite busy during this year’s ISCA, so I did not have the chance to read the papers or watch the videos carefully. I will try to read the ISCA 2021 papers that I am interested in, hopefull before MICRO 2021 starts.

[2020-07-15]Coterie: Exploiting Frame Similarity to Enable High-Quality Multiplayer VR on Commodity Mobile Devices

3 minute read

Published: July 15, 2020

This paper studies multiplayer Virtual Reality(VR) on mobile divice in ASPLOS 2020.

[2021-01-26] GraphACT: Accelerating GCN Training on CPU-FPGA Heterogeneous Platforms

1 minute read

Published: January 26, 2021

This paper is a CPU-FPGA heterogenrous platform for GCN training. CPU will do the communication intensive operations, and leave the computation intensive parts to CPU.

Background and Motivation

It is challenging to accelerate Graph Convolutional Networks because: (1) substantial and irregular data communication to propagate information within the graph (2) intensive computation to propagate information along the neural network layers (3) Degree-imbalance ofgraph nodes can significantly degrade the performance of feature propagation.

[2020-07-20]Optimus Prime: Accelerating Data Transformation in Servers

2 minute read

Published: July 20, 2020

This paper proposes an accelerator for Data Transformation (DT) in servers in ASPLOS 2020.

[2021-01-07] Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training

2 minute read

Published: January 07, 2021

This is a paper about GPU memory footprint reduction using selective recomputation during LSTM training process. Although I do not know much about LSTM network, this paper is quite clear and easy to understand.

[2020-07-26]High-Performance Deep-Learning Coprocessor Integrated into x86 SoC with Server-Class CPUs

2 minute read

Published: July 26, 2020

This paper is an industry-track paper in ASPLOS 2020. I used to read more acedemic paper than industry paper, so I lack the knowledge what the focus of the industry. This paper decribes Centaur Technology’s Ncore, the industry’s first high-performance DL coprocessor technology in-target into an x86 SoC with server class CPUs.

[2020-07-20]Optimus Prime: Accelerating Data Transformation in Servers

2 minute read

Published: July 20, 2020

This paper proposes an accelerator for Data Transformation (DT) in servers in ASPLOS 2020.

[2020-07-16]Wild and Crazy Ideas (ASPLOS 2020)

1 minute read

Published: July 16, 2020

Today I am not going to read a specific paper. Rather, I watch a talk about some innovative ideas and oppotunities for computer architecture. The vedio of the talk can be found here

[2020-07-15]Coterie: Exploiting Frame Similarity to Enable High-Quality Multiplayer VR on Commodity Mobile Devices

3 minute read

Published: July 15, 2020

This paper studies multiplayer Virtual Reality(VR) on mobile divice in ASPLOS 2020.

[2020-07-13]Orbital Edge Computing: Nanosatellite Constellations as a New Class of Computing System

1 minute read

Published: July 13, 2020

This is another best paper in ASPLOS 2020. “I think it’s the dawn of a new era in space exploration, I thinks a very exciting era,” said Elon Musk. So I am impressed by the edging computing in space.

[2020-07-12]Challenging Sequential Bitstream Processing via Principled Bitwise Speculation

1 minute read

Published: July 12, 2020

This is one of the best papers in ASPLOS 2020. I like the idea of transforming a somewhat unfamiliar problem into a equivalent well-solved problem.

[2020-07-20]Optimus Prime: Accelerating Data Transformation in Servers

2 minute read

Published: July 20, 2020

This paper proposes an accelerator for Data Transformation (DT) in servers in ASPLOS 2020.

[2021-04-20] DiAG: A Dataflow-Inspired Architecture for General-Purpose Processors

less than 1 minute read

Published: April 20, 2021

In my point of view, it is basically a spatial dataflow architecture with more general compatiblity.

[2020-07-13]Orbital Edge Computing: Nanosatellite Constellations as a New Class of Computing System

1 minute read

Published: July 13, 2020

This is another best paper in ASPLOS 2020. “I think it’s the dawn of a new era in space exploration, I thinks a very exciting era,” said Elon Musk. So I am impressed by the edging computing in space.

[2021-04-20] DiAG: A Dataflow-Inspired Architecture for General-Purpose Processors

less than 1 minute read

Published: April 20, 2021

In my point of view, it is basically a spatial dataflow architecture with more general compatiblity.

[2021-01-18] [CS224W] Graph Neural Network

less than 1 minute read

Published: January 18, 2021

Basics of Graph Neural Network

[2021-01-17] [CS224W] Graph Repesentation Learning

2 minute read

Published: January 17, 2021

Network embedding

Task: We map each node in a network into a low-dimensional space Goal: encode nodes so that similarity in the embedding space (e.g., dot product) approximates similarity in the original network.

[2021-01-15] [CS224W] Spectral Clustering

1 minute read

Published: January 15, 2021

Graph Partitioning

[2021-01-13] [CS224W] Community Structure in Networks

1 minute read

Published: January 13, 2021

Communities

[2021-01-12] [CS224W] Motifs and Structural Roles in Networks

1 minute read

Published: January 12, 2021

Subgraphs, Motifs

Network motifs: recurring, significant patterns of interconnections

• induced subgraphs - consider all edges connecting pairs of vertices in subset
• recurrence - allow overlapping of motifs
• signficance of a motif: Motifs are overrepresented in a network when compared to randomized networks

[2021-01-11] [CS224W] Properties of Networks and Random Graph Models

2 minute read

Published: January 11, 2021

I would like to learn GNN to see if there is any opportunity to optimize/accelerate it. But the surveying paper for GNN has too many expressions, notations which are hard to understand. So I will watch the CS224W Machine Learning with Graphs lecture to learn it in a smooth way. I will write down some key points I learned.

[2020-07-16]Wild and Crazy Ideas (ASPLOS 2020)

1 minute read

Published: July 16, 2020

Today I am not going to read a specific paper. Rather, I watch a talk about some innovative ideas and oppotunities for computer architecture. The vedio of the talk can be found here

[2021-01-18] [CS224W] Graph Neural Network

less than 1 minute read

Published: January 18, 2021

Basics of Graph Neural Network

[2021-01-17] [CS224W] Graph Repesentation Learning

2 minute read

Published: January 17, 2021

Network embedding

Task: We map each node in a network into a low-dimensional space Goal: encode nodes so that similarity in the embedding space (e.g., dot product) approximates similarity in the original network.

[2021-01-15] [CS224W] Spectral Clustering

1 minute read

Published: January 15, 2021

Graph Partitioning

[2021-01-13] [CS224W] Community Structure in Networks

1 minute read

Published: January 13, 2021

Communities

[2021-01-12] [CS224W] Motifs and Structural Roles in Networks

1 minute read

Published: January 12, 2021

Subgraphs, Motifs

Network motifs: recurring, significant patterns of interconnections

• induced subgraphs - consider all edges connecting pairs of vertices in subset
• recurrence - allow overlapping of motifs
• signficance of a motif: Motifs are overrepresented in a network when compared to randomized networks

[2021-01-11] [CS224W] Properties of Networks and Random Graph Models

2 minute read

Published: January 11, 2021

I would like to learn GNN to see if there is any opportunity to optimize/accelerate it. But the surveying paper for GNN has too many expressions, notations which are hard to understand. So I will watch the CS224W Machine Learning with Graphs lecture to learn it in a smooth way. I will write down some key points I learned.

[2021-01-07] Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training

2 minute read

Published: January 07, 2021

This is a paper about GPU memory footprint reduction using selective recomputation during LSTM training process. Although I do not know much about LSTM network, this paper is quite clear and easy to understand.

[2021-10-12] ISCA 2021 Section 3A: Machine Learning 1

1 minute read

Published: October 12, 2021

This reading blog is about three papers in Section 3A: Machine Learning 1 of ISCA 2021.

RaPiD: AI Accelerator for Ultra-Low Precision Training and Inference

This accelerator supports mixed precisions for both training and inference: 16 and 8-bit floating-point and 4 and 2-bit fixed point. It imporves both performance(TOPS) and energy efficiency(TOPS/W) at ultra-low preceision. In my opinion, this work contributes more on the engineering part (architecture).

[2021-09-24] Ten Lessons From Three Generations Shaped Google’s TPUv4i : Industrial Product

2 minute read

Published: September 24, 2021

I was quite busy during this year’s ISCA, so I did not have the chance to read the papers or watch the videos carefully. I will try to read the ISCA 2021 papers that I am interested in, hopefull before MICRO 2021 starts.

[2021-04-20] DiAG: A Dataflow-Inspired Architecture for General-Purpose Processors

less than 1 minute read

Published: April 20, 2021

In my point of view, it is basically a spatial dataflow architecture with more general compatiblity.

[2021-02-20] Analyzing and Mitigating Data Stalls in DNN Training

3 minute read

Published: February 20, 2021

Most DNN acclerator papers I read focus on DNN inference rather than training. From this paper, I learned that the bottleneck for DNN training is I/O for fetching data and CPU side for preprocessing.

Background

[2021-01-26] GraphACT: Accelerating GCN Training on CPU-FPGA Heterogeneous Platforms

1 minute read

Published: January 26, 2021

This paper is a CPU-FPGA heterogenrous platform for GCN training. CPU will do the communication intensive operations, and leave the computation intensive parts to CPU.

Background and Motivation

It is challenging to accelerate Graph Convolutional Networks because: (1) substantial and irregular data communication to propagate information within the graph (2) intensive computation to propagate information along the neural network layers (3) Degree-imbalance ofgraph nodes can significantly degrade the performance of feature propagation.

[2021-01-07] Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training

2 minute read

Published: January 07, 2021

This is a paper about GPU memory footprint reduction using selective recomputation during LSTM training process. Although I do not know much about LSTM network, this paper is quite clear and easy to understand.

[2020-11-14] High-Performance Deep-Learning Coprocessor Integrated into x86 SoC with Server-Class CPUs

3 minute read

Published: November 14, 2020

This is a paper using processor-in-memory(PIM) for NN acceleration in ASPLOS 2020. I know little about PIM techinques. And I don’t know much about the SRAM structure. I learned some basic knowledge of SRAM, PIM from this paper. And I learned a lot of the implementation tools and benchmark measuring tools.

[2020-07-26]High-Performance Deep-Learning Coprocessor Integrated into x86 SoC with Server-Class CPUs

2 minute read

Published: July 26, 2020

This paper is an industry-track paper in ASPLOS 2020. I used to read more acedemic paper than industry paper, so I lack the knowledge what the focus of the industry. This paper decribes Centaur Technology’s Ncore, the industry’s first high-performance DL coprocessor technology in-target into an x86 SoC with server class CPUs.

[2020-07-20]Optimus Prime: Accelerating Data Transformation in Servers

2 minute read

Published: July 20, 2020

This paper proposes an accelerator for Data Transformation (DT) in servers in ASPLOS 2020.

[2020-07-15]Coterie: Exploiting Frame Similarity to Enable High-Quality Multiplayer VR on Commodity Mobile Devices

3 minute read

Published: July 15, 2020

This paper studies multiplayer Virtual Reality(VR) on mobile divice in ASPLOS 2020.

[2020-07-13]Orbital Edge Computing: Nanosatellite Constellations as a New Class of Computing System

1 minute read

Published: July 13, 2020

This is another best paper in ASPLOS 2020. “I think it’s the dawn of a new era in space exploration, I thinks a very exciting era,” said Elon Musk. So I am impressed by the edging computing in space.

[2020-07-12]Challenging Sequential Bitstream Processing via Principled Bitwise Speculation

1 minute read

Published: July 12, 2020

This is one of the best papers in ASPLOS 2020. I like the idea of transforming a somewhat unfamiliar problem into a equivalent well-solved problem.

[2020-07-20]Optimus Prime: Accelerating Data Transformation in Servers

2 minute read

Published: July 20, 2020

This paper proposes an accelerator for Data Transformation (DT) in servers in ASPLOS 2020.

[2021-01-18] [CS224W] Graph Neural Network

less than 1 minute read

Published: January 18, 2021

Basics of Graph Neural Network

[2021-01-17] [CS224W] Graph Repesentation Learning

2 minute read

Published: January 17, 2021

Network embedding

Task: We map each node in a network into a low-dimensional space Goal: encode nodes so that similarity in the embedding space (e.g., dot product) approximates similarity in the original network.

[2021-01-15] [CS224W] Spectral Clustering

1 minute read

Published: January 15, 2021

Graph Partitioning

[2021-01-13] [CS224W] Community Structure in Networks

1 minute read

Published: January 13, 2021

Communities

[2021-01-12] [CS224W] Motifs and Structural Roles in Networks

1 minute read

Published: January 12, 2021

Subgraphs, Motifs

Network motifs: recurring, significant patterns of interconnections

• induced subgraphs - consider all edges connecting pairs of vertices in subset
• recurrence - allow overlapping of motifs
• signficance of a motif: Motifs are overrepresented in a network when compared to randomized networks

[2021-01-11] [CS224W] Properties of Networks and Random Graph Models

2 minute read

Published: January 11, 2021

I would like to learn GNN to see if there is any opportunity to optimize/accelerate it. But the surveying paper for GNN has too many expressions, notations which are hard to understand. So I will watch the CS224W Machine Learning with Graphs lecture to learn it in a smooth way. I will write down some key points I learned.