macbert4mdcspell_v3

macbert4mdcspell

概述(macbert4mdcspell)

• macro-correct, 中文拼写纠错CSC测评(文本纠错), 权重使用
• 项目地址在 https://github.com/yongzhuo/macro-correct
• 本模型权重为macbert4mdcspell v3, 使用mdcspell架构, 其特点是det label和cor label交互;
• 训练时加入了macbert的mlm-loss, 推理时舍弃了macbert后面的部分;
• 如何使用: 1.使用transformers调用; 2.使用macro-correct项目调用; 详情见***三、调用(Usage)***;
• 为了修复过纠问题, macbert4mdcspell v3的MFT只85%的时间no-error-mask(0.15), 5%的时间target-to-target, 10%的时间不mask;
• 训练数据为 2000w+, 相对于macbert4mdcspell v2, 强化了古诗和文言文的训练;

目录

一、测评(Test))

二、结论(Conclusion))

三、调用(Usage))

四、论文(Paper))

五、参考(Refer))

六、引用(Cite))

一、测评(Test)

• 1 测评数据来源

地址为Macropodus/csc eval public, 所有训练数据均来自公网或开源数据, 训练数据为1千万左右, 混淆词典较大;

• 2 测评数据预处理

• 3 其他说明

二、重要指标

• 1 F1(common cor f1)

model/common cor f1 avg gen de3 lemon v2 gen passage text proof gen xxqg faspell lomo tet mcsc tet ecspell sighan2013 sighan2014 sighan2015

:---------------------------------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :-----------------

shibing624/macbert4csc-base-chinese 45.8 42.44 42.89 31.49 46.31 26.06 32.7 44.83 27.93 55.51 70.89 61.72 66.81

shibing624/chinese-text-correction-1.5b 45.11 27.29 89.48 14.61 83.9 13.84 18.2 36.71 96.29 88.2 36.41 15.64 20.73

twnlp/ChineseErrorCorrector3-4B 53.59 30.28 89.43 22.94 39.9 16.89 30.53 71.0 99.92 72.43 65.02 47.81 56.88

relm v1 54.12 89.86 51.79 38.4 63.74 30.6 31.95 49.82 64.7 73.57 66.4 39.87 48.8

bert4csc v1 62.28 93.73 61.99 44.79 68.0 35.03 48.28 61.8 64.41 79.11 77.66 51.01 61.54

macbert4csc v1 68.55 96.67 65.63 48.4 75.65 38.43 51.76 70.11 80.63 85.55 81.38 57.63 70.7

macbert4csc v2 68.6 96.74 66.02 48.26 75.78 38.84 51.91 70.17 80.71 85.61 80.97 58.22 69.95

macbert4mdcspell v1 71.1 96.42 70.06 52.55 79.61 43.37 53.85 70.9 82.38 87.46 84.2 61.08 71.32

macbert4mdcspell v2 71.23 96.42 65.8 52.35 75.94 43.5 53.82 72.66 82.28 88.69 82.51 65.59 75.26

macbert4mdcspell v3 71.71 96.43 68.07 59.36 78.81 50.07 48.67 74.51 79.03 87.16 81.31 64.29 72.76

macbert4mdcspell v1 rethink2 69.64 92.4 67.99 57.69 77.49 50.38 53.96 69.35 84.65 88.26 70.96 56.05 66.54

macbert4mdcspell v2 rethink2 72.54 95.59 65.54 58.01 75.86 49.67 55.56 72.78 84.65 90.78 80.93 65.74 75.39

macbert4mdcspell v3 rethink2 71.82 95.05 67.48 62.19 78.0 55.2 49.5 74.26 81.72 87.56 76.75 62.96 71.12

• 2 acc(common cor acc)

model/common cor acc avg gen de3 lemon v2 gen passage text proof gen xxqg faspell lomo tet mcsc tet ecspell sighan2013 sighan2014 sighan2015

:---------------------------------------- :------ :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :----------------- :-----------------

shibing624/macbert4csc-base-chinese 48.26 26.96 28.68 34.16 55.29 28.38 22.2 60.96 57.16 67.73 55.9 68.93 72.73

shibing624/chinese-text-correction-1.5b 46.09 15.82 81.29 22.96 82.17 19.04 12.8 50.2 96.4 89.13 22.8 27.87 32.55

twnlp/ChineseErrorCorrector3-4B 51.85 17.87 81.2 27.32 48.17 23.44 20.8 77.16 99.92 76.6 49.0 47.18 53.55

relm v1 51.9 81.71 36.18 37.04 63.99 29.34 22.9 51.98 74.1 76.0 50.3 45.76 53.45

bert4csc v1 60.76 88.21 45.96 43.13 68.97 35.0 34.0 65.86 73.26 81.8 64.5 61.11 67.27

macbert4csc v1 65.34 93.56 49.76 44.98 74.64 36.1 37.0 73.0 83.6 86.87 69.2 62.62 72.73

macbert4csc v2 65.22 93.69 50.14 44.92 74.64 36.26 37.0 72.72 83.66 86.93 68.5 62.43 71.73

macbert4mdcspell v1 67.15 93.09 54.8 47.71 78.09 39.52 38.8 71.92 84.78 88.27 73.2 63.28 72.36

macbert4mdcspell v2 68.31 93.09 50.05 48.72 75.74 40.52 38.9 76.9 84.8 89.73 71.0 71.94 78.36

macbert4mdcspell v3 68.09 93.11 52.42 53.91 77.89 45.28 34.2 76.82 82.5 88.13 69.2 68.83 74.82

macbert4mdcspell v1 rethink2 65.04 85.88 52.42 51.69 76.23 44.52 38.9 70.78 86.48 88.93 55.8 59.98 68.91

macbert4mdcspell v2 rethink2 69.14 91.56 49.76 53.01 75.67 44.84 40.5 76.98 86.56 91.47 68.8 72.03 78.45

macbert4mdcspell v3 rethink2 67.84 90.57 51.76 56.24 77.19 49.16 34.9 76.64 84.4 88.47 63.1 67.98 73.64

• 3 acc(acc true, thr=0.75)

model/acc avg acc rmrb acc xxqg

:---------------------------------------- :------- :--------- :---------

shibing624/macbert4csc-base-chinese 99.24 99.22 99.26

shibing624/chinese-text-correction-1.5b 82.0 77.14 86.86

twnlp/ChineseErrorCorrector3-4B 77.03 76.96 77.1

relm v1 93.47 90.21 96.74

bert4csc v1 98.71 98.36 99.06

macbert4csc v1 97.72 96.72 98.72

macbert4csc v2 97.89 96.98 98.8

macbert4mdcspell v1 97.75 96.51 98.98

macbert4mdcspell v2 99.54 99.22 99.86

macbert4mdcspell v3 98.85 98.32 99.38

macbert4mdcspell v1 rethink2 92.78 88.31 97.24

macbert4mdcspell v2 rethink2 98.15 96.72 99.58

macbert4mdcspell v3 rethink2 98.85 98.32 99.38

数据集 alipayseq

model/common cor f1 alipayseq

:-------------------------------- :----------

shibing624/macbert4csc 15.36

twnlp/ChineseErrorCorrector3-4B 42.84

bert4csc v1 42.23

macbert4csc v1 48.45

macbert4csc v2 45.60

macbert4mdcspell v1 48.97

macbert4mdcspell v2 50.41

macbert4mdcspell v3 50.14

数据集 古诗-文言文数据集测评

model/common cor f1 det cor

:------------------------------------ :------ :------

shibing624/macbert4csc-base-chinese 44.12 7.48

macbert4mdcspell v1 58.98 12.43

macbert4mdcspell v2 50.61 10.40

macbert4mdcspell v3 73.24 47.41

二、结论(Conclusion)

三、调用(Usage)

• 1 使用macro-correct

• 2 使用 transformers

四、论文(Paper)

• 2024-Refining: Refining Corpora from a Model Calibration Perspective for Chinese
• 2024-ReLM: Chinese Spelling Correction as Rephrasing Language Model
• 2024-DICS: DISC: Plug-and-Play Decoding Intervention with Similarity of Characters for Chinese Spelling Check

• 2023-Bi-DCSpell: [A Bi-directional Detector-Corrector Interactive Framework for Chinese Spelling Check]()
• 2023-BERT-MFT: Rethinking Masked Language Modeling for Chinese Spelling Correction
• 2023-PTCSpell: PTCSpell: Pre-trained Corrector Based on Character Shape and Pinyin for Chinese Spelling Correction
• 2023-DR-CSC: A Frustratingly Easy Plug-and-Play Detection-and-Reasoning Module for Chinese
• 2023-DROM: Disentangled Phonetic Representation for Chinese Spelling Correction
• 2023-EGCM: An Error-Guided Correction Model for Chinese Spelling Error Correction
• 2023-IGPI: Investigating Glyph-Phonetic Information for Chinese Spell Checking: What Works and What’s Next?
• 2023-CL: [Contextual Similarity is More Valuable than Character Similarity-An Empirical Study for Chinese Spell Checking]()

• 2022-CRASpell: CRASpell: A Contextual Typo Robust Approach to Improve Chinese Spelling Correction
• 2022-MDCSpell: MDCSpell: A Multi-task Detector-Corrector Framework for Chinese Spelling Correction
• 2022-SCOPE: Improving Chinese Spelling Check by Character Pronunciation Prediction: The Effects of Adaptivity and Granularity
• 2022-ECOPO: The Past Mistake is the Future Wisdom: Error-driven Contrastive Probability Optimization for Chinese Spell Checking

• 2021-MLMPhonetics: Correcting Chinese Spelling Errors with Phonetic Pre-training
• 2021-ChineseBERT: ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information
• 2021-BERTCrsGad: Global Attention Decoder for Chinese Spelling Error Correction
• 2021-ThinkTwice: Think Twice: A Post-Processing Approach for the Chinese Spelling Error Correction
• 2021-PHMOSpell: PHMOSpell: Phonological and Morphological Knowledge Guided Chinese Spelling Chec
• 2021-SpellBERT: SpellBERT: A Lightweight Pretrained Model for Chinese Spelling Check
• 2021-TwoWays: Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models
• 2021-ReaLiSe: Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking
• 2021-DCSpell: DCSpell: A Detector-Corrector Framework for Chinese Spelling Error Correction
• 2021-PLOME: PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction
• 2021-DCN: Dynamic Connected Networks for Chinese Spelling Check

• 2020-SoftMaskBERT: Spelling Error Correction with Soft-Masked BERT
• 2020-SpellGCN: SpellGCN：Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check
• 2020-ChunkCSC: Chunk-based Chinese Spelling Check with Global Optimization
• 2020-MacBERT: Revisiting Pre-Trained Models for Chinese Natural Language Processing

• 2019-FASPell: FASPell: A Fast, Adaptable, Simple, Powerful Chinese Spell Checker Based On DAE-Decoder Paradigm
• 2018-Hybrid: A Hybrid Approach to Automatic Corpus Generation for Chinese Spelling Checking

• 2015-Sighan15: Introduction to SIGHAN 2015 Bake-off for Chinese Spelling Check
• 2014-Sighan14: Overview of SIGHAN 2014 Bake-off for Chinese Spelling Check
• 2013-Sighan13: Chinese Spelling Check Evaluation at SIGHAN Bake-off 2013

五、参考(Refer)

• nghuyong/Chinese-text-correction-papers
• destwang/CTCResources
• wangwang110/CSC
• chinese-poetry/chinese-poetry
• chinese-poetry/huajianji
• garychowcmu/daizhigev20
• yangjianxin1/Firefly
• Macropodus/xuexiqiangguo 428w
• Macropodus/csc clean wang271k
• Macropodus/csc eval public
• shibing624/pycorrector
• iioSnail/MDCSpell pytorch
• gingasan/lemon
• Claude-Liu/ReLM

六、引用(Cite)

For citing this work, you can refer to the present GitHub project. For example, with BibTeX: