HYEONG

Naver AI Boostcamp Diary

부스트캠프 11주차 주간학습 정리

0. 배운 것들 Advanced Topis in MRC Elasticsearch Sources of Bias in MRC

부스트캠프 10주차 주간학습 정리

0. 배운 것들 일주일 복습

[AI Boostcamp Day 36] Special Lectures

Career as a ML Developer? ML modeling is getting more and more automated. Should learn other technical stacks as well. FE abilities for making ...

[AI Boostcamp Day 34]

What did I learn today self supervised pretraining models

부스트캠프 6주차 주간학습 정리

0. 일간 학습 정리 링크 수요일 (210915) 금요일 (210917)

[AI Boostcamp Day 32]

What did I learn today Finished the note for transformer Researched Hugging Face Researched Baseline Models to Benchmark for our competition ...

부스트캠프 5주차 주간학습 정리

0. 일간 학습 정리 링크 월요일 (210906) 화요일 (210907) 목요일 (210909)

[AI Boostcamp Day 28] Seq2Seq, Beam Search, and BLEU Score

What did I learn today Seq2Seq Image Source Takes a sequence of words as input and gives another sequence as an output Composed ...

[AI Boostcamp Day 26] Basic RNN, LSTM, GRU

What did I learn today RNN image source : prior hidden-state vector input vector at time t current hidden-state vector ...

부스트캠프 Meet Up

1. 부스트캠프는 만능이 아님 코딩테스트, 면접 준비 등 취업을 하려면 추가적으로 해야 하는 것들이 있다 다만 부스트캠프에는 함께 준비할 사람들이 있다는 것! 나중에 준비해도 되지만, 미리 준비하면 편하게 갈 수 있다!

[AI Boostcamp Day 25] Word Embedding, Basic Concept of NLP

What did I learn today Academic Disciplines related to NLP NLP Low Level Parsing Tokenization, st...

[AI Boostcamp Day 21] Training, Training and Training

What did I do today Found wrong photos (labelled wrong from the first place) -> revised and distributed so that the teammates could use Found out tha...

[AI Boostcamp Day 20] Training, Training and Training

What did I do today Stratified labeling (single person either in train or test dataset) Served transformation package

[AI Boostcamp Day 19] Ensemble and Training Tools

Ensemble Combining multiple models for better performance Voting Hard Voting Soft Voting Cross Validation The fold...

부스트캠프 4주차 주간학습 정리

0. 일간 학습 정리 링크 월요일 (210823) 화요일 (210824) 수요일 (210825) 목요일 (210826) 금요일 (210827)

[AI Boostcamp Day 18] Training and Inference

Training and Inference Blocks for Training Loss Loss function == Error function -> Used for backpropagation Fam...

[AI Boostcamp Day 17] Model

Model The building blocks of a PyTorch model, nn.Modules are pythonicly implementable Pretrained Model Apply transfer learning Big Data...

[AI Boostcamp Day 16] Dataset and Data Generation

What I Learned Today Pre-processing 80% time of the project Image Dataset bounding box: sometimes to remove noise ...

[AI Boostcamp Day 15] P Stage 시작

P Stage 시작 V100 서버 제공 첫 경진대회: CV, 마스크 착용여부 예측 팁 게시판, 랜덤 게시판 등으로 공유 권장 F1 Score

[AI Boostcamp Day 14] All about the Training Process

What I studied today: Multi-GPU Training Huge Data -> need of powerful hardware GPU vs Node GPU is a card Node i...

부스트캠프 3주차 주간학습 정리

0. 일간 학습 정리 링크 화요일 (210817) 수요일 (210818) 목요일 (210818) 금요일 (210818)

[AI Boostcamp Day 13] Saving, Transfering and Monitoring Models

What I studied today: Recent Tendency of Using Models Using pretrained models to fine-tune on a specific task

AI Boostcamp Day 12

What I studied today: AutoGrad & Optimizer nn.Parameter Inherits tensor object If an attribute of nn.Module, required...

AI Boostcamp Day 11

What I studied today: Pytorch vs Tensorflow Computatioanl Graph Creating a graph used when computing training & predicti...

부스트캠프 2주차 주간학습 정리

0. 일간 학습 정리 링크 월요일 (210809) 화요일 (210810) 수요일 (210811) 목요일 (210811) 금요일 (210811)

AI Boostcamp Day 10

What I studied today: Generative Model? Not only confined to generation of images or videos Classification model is a form of generative ...

AI Boostcamp Day 9

What I studied today: Sequential Model

AI Boostcamp Day 8

What I studied today: How Kernerls Work Input size, kernel size, output size Knowing only two above the above, can tell the size of the o...

AI Boostcamp Day 7

What Constitutes an Optimization Generalization: Training the network with a train data to perform well on an actual data. Generalization Gap: ...

AI Boostcamp Day 6

What I studied today: History of Deep Learning AlexNet (2012): first to prove the performance of neural networks. Outperformed conventional statistical ma...

부스트캠프 1주차 주간학습 정리

0. 일간 학습 정리 링크 월요일 (210802) 화요일 (210803)

DL Short Notes

[PyTorch Syntax] Tensor Operations

Pytorch Tensor operators Numpy + AutoGrad Tensor list (Python) -> Ndarray (Numpy) -> Tensor (PyTorch) Almost identical with Ndarr...

데이터 시각화

0. 들어가기에 앞서 아래 강의는 안수빈 서울대학교 컴퓨터공학과 석박과통합과정 학생의 부스트 캠프 강의를 요약하여 개인적인 첨언을 담은 포스트입니다

Transformer Short Note

Tramsformer: To solve the problem in dealing with sequential data; the input often changes in sequence and length Recursive x attententive o Encoder (...

Sequential Short Note

Sequential Model: Problem: does not know about the dimension of the input Naive Sequence Model: Consider all information from the past. Probl...

MRC

Source of Bias in MRC

1. Mitigating Training Bias A lot of bias present; models can learn sexual bias

Advanced Topis in MRC

1. Closed-book QA Popular approaches such as MRC and Open-domain QA are allowed to use supporting documents outside of the model New approaches have tri...

Elastic Search Summary

1. Types of Retrieval Boolean Retrieval: retrieves a document if it includes specified word Rank Retrieval: specify a weight for terms. The weight for w...

MRC Summary

1. What is MRC

Deep Learning Visualization

Basic Weight and Biases with Code

Weight and Biases Easy to share the experiemnt with other people! can track hardware status install with pip !pip install wandb -q make ...

Basic Tensorboard with Code

Tensorboard Supports PyTorch Shows scalars (accuracy, loss etc) Shows the computational graph Shows the histogram of weights

NLP

NLP - Unsupervised Pretraining Models

Contents

NLP - Transformer

Contents

Math

부스팅 (Boosting) 개념 이해하기 쉬운 수식 정리

Boosting 은 모델을 순차적으로 결합하는 모델 합성 방식이다. 첫 부스팅 알고리즘인 AdaBoost가 등장한 지 20년이 넘었기 때문에 구체적인 알고리즘은 모델마다 크게 다르지만, 순차적으로 이전 모델들의 오답에 유의하여 새 모델을 학습하는 아이디어는 동일하다.

Paper Review

[Paper Review] ViT (Vision Transformer)

Objective Paper: AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE

Competition

Boostcamp First CV Competition Abstract

Goal Classify 18 agumented classes 3 age intervals mask, no mask, and incorrect mask male or female

Posts by Category