Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
239 commits
Select commit Hold shift + click to select a range
e521d38
Upload all 자료팀
Aug 17, 2021
d71ea5f
app.py conflict 해결
Aug 17, 2021
2ecc9dd
add userlocallibmecab in .gitignore
Aug 17, 2021
2004af4
gitignore modified(add BornToBeLoved)
BornToBeLoved Aug 18, 2021
086e70b
first setting for dapi1
BornToBeLoved Aug 18, 2021
4324ac3
changed cen_dict from (idx:word) to (word:centrality)
BornToBeLoved Aug 18, 2021
31190a3
modified import
Aug 18, 2021
b0d42ae
modified app.py conflict resolved
BornToBeLoved Aug 18, 2021
4970d72
SVM에서 필요한 디렉토리와 파일이 없을 때 생성하는 함수 업데이트
Aug 21, 2021
a3f4452
CNN, Multi_SVM의 train시 필요한 디렉토리, 폴더(log, train_data)가 없을 때 만들어 주는 함수 추가
Aug 21, 2021
eacb7b8
CNN.py에서 make_dir(주제 알고리즘 실행시 필요한 디렉토리와 폴더 만들어 주는 함수)의 파일 이름 수정(기존에는 …
Aug 21, 2021
b131d15
remove unnecessary library
BornToBeLoved Aug 22, 2021
333268e
remove testline
BornToBeLoved Aug 22, 2021
a64ffa5
logger setting
BornToBeLoved Aug 22, 2021
bc3c74c
add analyzer(kmeans)
BornToBeLoved Aug 22, 2021
1cca656
kmean analyzer (before jsonify the result)
BornToBeLoved Aug 22, 2021
47ea9c7
kmeans json return
BornToBeLoved Aug 23, 2021
f0efeb6
json return debugging(int type error)
BornToBeLoved Aug 23, 2021
34bb865
edit .gitignore
BornToBeLoved Aug 23, 2021
27dc6dc
merge middeware(add kmeans and logger)
BornToBeLoved Aug 23, 2021
2f72d3a
debug logger
BornToBeLoved Aug 23, 2021
17f68f5
debug logger
BornToBeLoved Aug 23, 2021
c419a6c
Merge branch 'middleware'
BornToBeLoved Aug 23, 2021
5d0230e
fix logger
BornToBeLoved Aug 23, 2021
7039e56
fix logger(to merge with master)
BornToBeLoved Aug 23, 2021
d16ff4a
Merge branch 'middleware'
BornToBeLoved Aug 23, 2021
6c7e495
import error
BornToBeLoved Aug 24, 2021
0fb52cf
Merge branch 'middleware'
BornToBeLoved Aug 24, 2021
b3cffad
tfidf debug
Aug 24, 2021
70de055
kmeans debug
BornToBeLoved Aug 24, 2021
3bf0fae
Merge branch 'middleware'
BornToBeLoved Aug 24, 2021
749a779
count retunr change, kmeans parameter change
BornToBeLoved Aug 26, 2021
b3118ad
log file generate, and linked loggger(in kmeans) to flask.app
BornToBeLoved Aug 26, 2021
bc57844
little change(remove test code, edit gitignore)
BornToBeLoved Aug 26, 2021
bdea323
output setting
BornToBeLoved Aug 27, 2021
733d71a
logging-debug error fix
Aug 31, 2021
c460861
logging debug fix-2
Aug 31, 2021
81b5e94
json return (mystorate)
BornToBeLoved Sep 1, 2021
7ef5cc0
getCount return error fix
Sep 1, 2021
89a0be2
adapt optionlist to network analysis
BornToBeLoved Sep 1, 2021
4cd01de
test log fix
BornToBeLoved Sep 1, 2021
b752de2
network mongodb fix
BornToBeLoved Sep 1, 2021
f1c0176
Merge branch 'master' into middleware
BornToBeLoved Sep 2, 2021
22dc196
kmeans clusternum error fix
BornToBeLoved Sep 2, 2021
0ea7643
kmeans clusterNum error fix
BornToBeLoved Sep 2, 2021
475abe8
clusterNum error fix
BornToBeLoved Sep 5, 2021
9029be1
extract top_words by ngram frequency
BornToBeLoved Sep 5, 2021
f3bed3d
hcluster add
BornToBeLoved Sep 5, 2021
135c624
add ngram and hcluster
BornToBeLoved Sep 5, 2021
11d42e3
Merge branch 'master' of https://github.com/KUBIC-HGU/TIBigdataMiddle…
BornToBeLoved Sep 5, 2021
d157838
hcluster mongodb update
BornToBeLoved Sep 5, 2021
cd8063f
app.py kmeans clusterhum type error --> chanted from string to int, …
Sep 9, 2021
61b7fd6
deleted document error fix
Sep 10, 2021
f244f07
preprocessing logging added
Sep 10, 2021
84a10f9
kmeans option parameter change
BornToBeLoved Sep 13, 2021
3d5ab5f
adjust linkStrength option at network analysis
BornToBeLoved Sep 13, 2021
fc48f37
add linkStrength option in sematic network analysis alrorithm
BornToBeLoved Sep 13, 2021
a2406c1
remove testcode
BornToBeLoved Sep 13, 2021
794b024
fix requested data name
Sep 13, 2021
b32b743
added text information in network analysis
BornToBeLoved Sep 14, 2021
73695a4
Merge branch 'middleware'
BornToBeLoved Sep 14, 2021
3f88db2
주석처리 misisng
BornToBeLoved Sep 14, 2021
601392e
Logging - preprocessing
Sep 15, 2021
747cb7a
Added preprocessing logging and returning errMsg
Sep 23, 2021
22a2de4
add loggging and return errMsg(wordCount)
Sep 23, 2021
1e2ecf8
add loggging and return errMsg(tfidf)
Sep 23, 2021
3f1a9f3
add logging and return errMsg(kmeans)
Sep 23, 2021
7fec1eb
add logging and returning errMsg (kmeans)
Sep 23, 2021
d2cd766
identification error fixed
BornToBeLoved Oct 12, 2021
aa9dfe4
tfidf - sorting result
BornToBeLoved Oct 14, 2021
65934d8
add wordCount analysis where include wordcount result
BornToBeLoved Oct 15, 2021
756ea18
add input type check
BornToBeLoved Oct 15, 2021
d4a1125
add input type check(tfidf)
BornToBeLoved Oct 27, 2021
def8602
add input type check(semanticNetworkAnalysis)
BornToBeLoved Oct 27, 2021
328d3ae
add input type check(wordCount)
BornToBeLoved Oct 27, 2021
77b0b82
fix logging (sementicNetworkAnalysis)
BornToBeLoved Oct 27, 2021
06e0c91
fix logging (tfidf)
BornToBeLoved Oct 27, 2021
410523d
add input type check(kmeans)
BornToBeLoved Oct 27, 2021
20ff2fd
mongodb find latest doc find_one --> find.sort.limit(1)
BornToBeLoved Oct 28, 2021
e0083ec
added exeption handling
BornToBeLoved Oct 28, 2021
8f3a165
fixed type error: option 1, 2 str to int
BornToBeLoved Oct 28, 2021
9ee3fd0
fixed error msg:
BornToBeLoved Oct 28, 2021
a44a8ea
changed return form(preprocessin)
BornToBeLoved Oct 28, 2021
cc1c4d7
added exception handling(hcluster)
BornToBeLoved Oct 28, 2021
40802e5
minor changes
BornToBeLoved Nov 17, 2021
fa434f3
changed print option to write 2021 report
BornToBeLoved Dec 26, 2021
57d8c12
changed example code
BornToBeLoved Dec 26, 2021
104d061
added debugging code(log)
BornToBeLoved Dec 26, 2021
8698fbf
changed print option to write 2021 report
BornToBeLoved Dec 26, 2021
f2f16e0
changed err message(where ncluster > ndoc)
BornToBeLoved Dec 26, 2021
28f92fc
fixed getCount error
BornToBeLoved Dec 26, 2021
632190b
fixed graph not connected error
BornToBeLoved Dec 26, 2021
83e8ee3
add topic modeling(LDA)
BornToBeLoved Jan 12, 2022
87270bd
add word embedding(word2vec)
BornToBeLoved Jan 12, 2022
65d9c80
log changed
BornToBeLoved Jan 12, 2022
2014d35
topicLDA added in app.py
BornToBeLoved Jan 12, 2022
53c8ddc
interaction with anular(LDA, W2V)
BornToBeLoved Jan 14, 2022
da5f847
error fix(deleted test code)
BornToBeLoved Jan 24, 2022
4feea18
add gitignore
BornToBeLoved Jan 24, 2022
0c859cf
added gitignore
BornToBeLoved Jan 24, 2022
bd8676f
added gitignore
BornToBeLoved Jan 24, 2022
1e59c51
removed git cached to apply gitignore
BornToBeLoved Jan 24, 2022
c62e589
removed git cached to apply gitignore
BornToBeLoved Jan 24, 2022
235af4f
deleted .html file
BornToBeLoved Jan 24, 2022
851a92b
added gitignore
BornToBeLoved Jan 24, 2022
7f97eed
[FIX] connecting es using SSL without verify_certs=False
BornToBeLoved Jan 24, 2022
7bb5486
[FIX] [FIX] connecting es using SSL without verify_certs=False
BornToBeLoved Jan 24, 2022
0420224
[SECURITY] mongodb account info
BornToBeLoved Jan 26, 2022
b96bbd4
Merge branch 'master' into BE_Sungwon
BornToBeLoved Jan 26, 2022
eba9240
[FEAT] add return information(jsonDocId and analysisdate) in analysis
BornToBeLoved Mar 3, 2022
5bd38cd
Merge branch 'master' into BE_Sungwon
BornToBeLoved Mar 3, 2022
bd71442
[REFACTORING] Changed preprocessing. Add sentence information to resulte
BornToBeLoved Mar 18, 2022
3593666
[TEST] cmm, prs, relatedDoc_all
BornToBeLoved Mar 18, 2022
d4226dc
[fix] hide ip and port information
BornToBeLoved Mar 21, 2022
e4221bc
[FIX] changed mecab instantiation option
BornToBeLoved Mar 21, 2022
7ed0aed
[FEAT] able to analyze 3 dimension preprocessed doc
BornToBeLoved Mar 22, 2022
290a299
[TEST] deleted test code
BornToBeLoved Mar 22, 2022
32c8caf
[TEST] changed testcode
BornToBeLoved Mar 28, 2022
5177a93
[FEAT] enable to analyze 3 dimension preprocessed-list
BornToBeLoved Mar 28, 2022
242c56c
[TEST] added 3 dimension preprocessed-list test code
BornToBeLoved Mar 28, 2022
46edf99
[TEST] fixed 3 dimension preprocessed-list test code
BornToBeLoved Mar 28, 2022
0cd6454
[FEAT] added save option
BornToBeLoved Mar 28, 2022
f4fe353
[FIX] fixed save option
BornToBeLoved Mar 28, 2022
d99e057
[FEAT] able to analyze 3 dimension preprocessed doc
BornToBeLoved Mar 29, 2022
50cdccb
[FEAT] able to analyze 3 dimension preprocessed doc
BornToBeLoved Mar 29, 2022
e418eec
[FEAT] able to analyze 3 dimension preprocessed doc
BornToBeLoved Mar 29, 2022
f629787
[FEAT] able to analyze 3 dimension preprocessed doc
BornToBeLoved Mar 29, 2022
1a22199
[TEST] test code remove
BornToBeLoved Mar 29, 2022
13c3486
[FEAT] able to analyze 3 dimension preprocessed doc
BornToBeLoved Mar 29, 2022
f584b4b
[FIX] unusing variable removed
BornToBeLoved Mar 29, 2022
86fa694
[ADD] add count info to node
BornToBeLoved Mar 29, 2022
397aa14
[REFACTORING] hidden the IP and PORT number
BornToBeLoved Mar 29, 2022
acdcd3d
[DOCS] add relatedDoc related file
BornToBeLoved Mar 29, 2022
e42a550
file changed
BornToBeLoved Mar 29, 2022
f6cafc6
[FIX] fixed by someone
BornToBeLoved Mar 29, 2022
f39f404
[DOCS] added explanation
BornToBeLoved Mar 29, 2022
2a170ec
merge
BornToBeLoved Mar 29, 2022
475e3ae
[FIX] deleted unknown module
BornToBeLoved Mar 29, 2022
e2e0d0f
[FIX] debug mode off
BornToBeLoved Mar 30, 2022
28a2e6b
[FIX] fixed preprocessing return data
BornToBeLoved Mar 30, 2022
0112186
[FIX] fixed preprocessing return function
BornToBeLoved Mar 30, 2022
1d84977
[FIX] removed test code
BornToBeLoved Mar 30, 2022
e27e029
[FIX] removed test code
BornToBeLoved Mar 30, 2022
39756b3
Delete esAccount.pyc
BornToBeLoved Mar 30, 2022
56bbce0
[FIX] able to train svm model
BornToBeLoved Mar 30, 2022
9e37c53
[FEAT] rcmd
testation21 Apr 1, 2022
c90222d
취소해라
BornToBeLoved Apr 8, 2022
50459af
log changed
BornToBeLoved Apr 18, 2022
a57ec06
[FEAT] changed db name for save SVM result
BornToBeLoved Apr 20, 2022
6b6f3fd
Merge branch 'master' of https://github.com/KUBIC-HGU/TIBigdataMiddle…
BornToBeLoved Apr 20, 2022
48808d1
[FIX] able to apply user dict in preprocess
BornToBeLoved Apr 21, 2022
e6b4b3f
Merge branch 'master' of https://github.com/KUBIC-HGU/TIBigdataMiddle…
BornToBeLoved Apr 21, 2022
2eeef03
Merge branch 'BE_Sungwon'
BornToBeLoved Apr 21, 2022
a2cd82f
[FIX] able to apply user dict in preprocess
BornToBeLoved Apr 21, 2022
83b21b2
Merge branch 'master' of https://github.com/KUBIC-HGU/TIBigdataMiddle…
BornToBeLoved Apr 21, 2022
c1e75a4
[DOCS] 주석 수정
BornToBeLoved Apr 21, 2022
4dca4f4
[TEST] added test code
BornToBeLoved Apr 21, 2022
84a97bf
[FIX] able to apply compound dict to preprocessing
BornToBeLoved Apr 27, 2022
cd58f2e
Merge branch 'master' into BE_Sungwon
BornToBeLoved Apr 27, 2022
17d7208
merge with master
BornToBeLoved Apr 27, 2022
9715fe9
[FIX] changed tfidf to wordcount analysis
BornToBeLoved Apr 27, 2022
71f17ea
[FIX] able to apply default compound dict
BornToBeLoved Apr 27, 2022
3431e89
[FIX] changed tfidf_all to wordCount_all
BornToBeLoved Apr 29, 2022
08900f9
[FIX] SVM schedulars run 1st day of each months
BornToBeLoved Apr 29, 2022
8177f52
[FIX] linkStrength err fixed
BornToBeLoved Apr 29, 2022
f71b8a9
[FIX] logger err fixed
BornToBeLoved Apr 29, 2022
bfc87b7
[FIX] fixed filter function(filter by percentile)
BornToBeLoved May 10, 2022
2d2dd91
[FIX] removed SVM train scheduler
BornToBeLoved May 10, 2022
5f36c68
[FIX] test line deleted
BornToBeLoved May 10, 2022
8357fb9
[FIX] added code that excute without scheduler
BornToBeLoved May 10, 2022
8056ec7
[DOCS] SVM log changed
BornToBeLoved May 10, 2022
3038db4
[FIX] added post_body file
BornToBeLoved May 10, 2022
bab83f6
[FIX] added testmode and changed vectorizer setting
BornToBeLoved May 12, 2022
16a891a
[FIX] added testmode and changed vectorizer setting
BornToBeLoved May 12, 2022
8602829
[DOCS] gitignore and logfile
BornToBeLoved May 12, 2022
37ae196
[FIX] fixed return item num error
BornToBeLoved May 12, 2022
b794497
[FIX] added linkedEdgeIDList None type case
BornToBeLoved May 12, 2022
3110762
Merge remote-tracking branch 'origin/rcmd' into BE_Sungwon
BornToBeLoved May 12, 2022
d662876
[FIX] changed os path from FE to Middleware
BornToBeLoved May 12, 2022
97dc625
[FIX] changed data type to reduce memory
BornToBeLoved May 12, 2022
be90b50
[FIX] fixed file location error
BornToBeLoved May 12, 2022
c041bf9
[FIX] fixed NaN type error, added logger
BornToBeLoved May 12, 2022
a548747
file rearrange
BornToBeLoved May 12, 2022
8a55510
[FIX] removed outdated code
BornToBeLoved May 12, 2022
82ee8b1
[FIX] changed tag filtering algorithm in preprocessing
BornToBeLoved May 12, 2022
d10cf65
[FIX] added NNBC tag
BornToBeLoved May 12, 2022
175f724
[FIX] python grammar error fixed
BornToBeLoved May 12, 2022
ea400b6
[FIX] changed docID key to hashKey
BornToBeLoved May 13, 2022
4eb5c77
[FIX] able to analyze bigdata
BornToBeLoved May 13, 2022
69f47e5
[FIX] analyze all doc in ES group by 10000 docs
BornToBeLoved May 14, 2022
ce9fbfc
[FIX] changed mongo function insert --> insert_one
BornToBeLoved May 14, 2022
cdd22d1
[FIX] added target text (postbody) in preprocessing
BornToBeLoved May 16, 2022
cfdcc8a
[FIX] added count data to node information in ngrams analysis
BornToBeLoved May 16, 2022
40323e3
[FEAT] added all word option which allows count all word in doc
BornToBeLoved May 16, 2022
412d2b3
[FIX] return count info to result(node)
BornToBeLoved May 18, 2022
d10a4e0
[FIX] changed precentile func
BornToBeLoved May 18, 2022
d4680ba
[FIX] changed precentile func
BornToBeLoved May 18, 2022
a57dba2
[FIX] if network is not connected, get a largest network
BornToBeLoved May 18, 2022
cc46a4c
[FEAT] How to use WSGI server!
BornToBeLoved May 18, 2022
d245a76
[FEAT] added code which can save text and result as .txt and .csv file
BornToBeLoved May 18, 2022
cad2dff
[FIX] deleted print function
BornToBeLoved Jun 9, 2022
e4d9e1f
[FIX] changed index and makeCorpus func
BornToBeLoved Jun 9, 2022
4406fe7
[FIX] changed file name
BornToBeLoved Jun 13, 2022
ad7c674
[FIX] fixed empty hashkey error
BornToBeLoved Jun 13, 2022
245daaa
[FIX] changed(added) target ES index
BornToBeLoved Jun 13, 2022
4877f8f
[FIX] fixed ssl error and file none type error
BornToBeLoved Jul 5, 2022
a65a7ae
[FIX] removed unusing router (/svm, /train)
BornToBeLoved Jul 5, 2022
30f6f3c
[FEAT] able to analyze engilsy doc
BornToBeLoved Jul 5, 2022
2394a8c
[DOCS] removed unusing code
BornToBeLoved Jul 5, 2022
99f1c97
[FEAT] able to classify and preprocess the english doc.
BornToBeLoved Jul 5, 2022
436d40a
[DOCS] log updated
BornToBeLoved Jul 5, 2022
a129aa8
[TEST] connection test
BornToBeLoved Jul 5, 2022
90fff77
[FEAT] 사용자사전용 디렉토리 추가 함수 생성
BornToBeLoved Jul 5, 2022
c24d3ba
[FEAT] 사용자사전폴더에 초기사전 설치
BornToBeLoved Jul 5, 2022
ffebdd0
[FIX] 사용자사전 파일이 이미 있는경유에도 설치하는 오류 해결
BornToBeLoved Jul 5, 2022
2e04cba
[FEAT] 전처리시 사용자별 폴더에 저장된 사용자사전을 적용.
BornToBeLoved Jul 5, 2022
289ff38
[FIX] mecab 사용자사전 컴파일 오류 해결
BornToBeLoved Jul 5, 2022
42dee3c
[FEAT] 사용자별 사용자사전기능 업데이트
BornToBeLoved Jul 5, 2022
f8d1dde
Merge branch 'BE_Sungwon'
BornToBeLoved Jul 5, 2022
e34bbcf
[TEST] removed testcode
BornToBeLoved Jul 5, 2022
0814168
Merge branch 'BE_Sungwon'
BornToBeLoved Jul 5, 2022
ab3aac1
[FIX] 사용자사전 적용 오류 발생. 업데이트 전으로 롤백.
BornToBeLoved Jul 5, 2022
61cac7e
Merge branch 'BE_Sungwon'
BornToBeLoved Jul 5, 2022
a0fbbf4
[FIX] user-dic application error fixed
BornToBeLoved Jul 5, 2022
72bef23
Merge branch 'BE_Sungwon'
BornToBeLoved Jul 5, 2022
37a06d1
[TEST] test code removed
BornToBeLoved Jul 5, 2022
d226dcd
Merge branch 'BE_Sungwon'
BornToBeLoved Jul 5, 2022
63e3a0c
[FEAT] add mecab-ko-dict first compile check
BornToBeLoved Jul 14, 2022
dbbfe3c
[FEAT] Check whether compilation is required
BornToBeLoved Jul 14, 2022
ea9ffe6
[FEAT] added chinese character in preprocessing
BornToBeLoved Jul 14, 2022
f45e67e
[FIX] Fixed non-file error (my_dict.csv)
BornToBeLoved Jul 14, 2022
71b1f05
[DOCS] changed log code
BornToBeLoved Jul 14, 2022
4925ab1
[FEAT] added english stopwords prosess
BornToBeLoved Jul 14, 2022
76b2603
[FIX] deleted unused code(about log)
BornToBeLoved Nov 2, 2022
fc5b964
[FIX] fixed debug code and added some explanation
BornToBeLoved Nov 2, 2022
4147309
[FEAT] able to analyze hanja
BornToBeLoved Nov 2, 2022
96a53db
[DOCS] added comments
BornToBeLoved Nov 2, 2022
d8d06b5
[FEAT] added all index option
BornToBeLoved Nov 2, 2022
12e7730
[DOCS] log added
BornToBeLoved Nov 2, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
### 아래와 같은 확장자 무시 default ###
**/*.csv
**/*.jpg
**/*.json
**/*.pickle
**/esAccount.py
**/__pycache__/*
**/**/esAccount.py
**/**/__pycache__/*

### HaEunMok ###
model/
env/
train_data/
topic_analysis/esAccount.py
topic_analysis/MongoAccount.py
topic_analysis/__pycache__/
testdisk.log

### byk ###
TextMining/__pycache__/
mecab-0.996-ko-0.9.2/
mecab-ko-dic-2.1.1-20180720/
TextMining/userlocallibmecab/
TextMining/Analyzer/__pycache__/
TextMining/Tokenizer/__pycache__/
TextMining/Tokenizer/esAccount.py
.viminfo
kubic_sslFile.py
tib_topic_model

### baek ###
tfidfs/

### BornToBeLoved ###
mecab-python-0.996/
log_flask/
whatap/
__pycache__/
TextMining/userlocallibmecab/
account/
log/
common/
Labs/

### relatedDoc ###
test_cpu_result.txt
test_rcmd_cpu.sh
rcmd/log/
Loading