使用预训练的词嵌入

目录数量: 20
目录名称: ['comp.sys.ibm.pc.hardware', 'comp.os.ms-windows.misc', 'comp.windows.x', 'sci.space', 'sci.crypt', 'sci.med', 'alt.atheism', 'rec.autos', 'rec.sport.hockey', 'talk.politics.misc', 'talk.politics.mideast', 'rec.motorcycles', 'talk.politics.guns', 'misc.forsale', 'sci.electronics', 'talk.religion.misc', 'comp.graphics', 'soc.religion.christian', 'comp.sys.mac.hardware', 'rec.sport.baseball']
comp.graphics 中的文件数量: 1000
一些示例文件名: ['39638', '38747', '38242', '39057', '39031']

Newsgroups: comp.graphics
Path: cantaloupe.srv.cs.cmu.edu!das-news.harvard.edu!noc.near.net!howland.reston.ans.net!agate!dog.ee.lbl.gov!network.ucsd.edu!usc!rpi!nason110.its.rpi.edu!mabusj
From: mabusj@nason110.its.rpi.edu (Jasen M. Mabus)
Subject: Looking for Brain in CAD
Message-ID: <c285m+p@rpi.edu>
Nntp-Posting-Host: nason110.its.rpi.edu
Reply-To: mabusj@rpi.edu
Organization: Rensselaer Polytechnic Institute, Troy, NY.
Date: Thu, 29 Apr 1993 23:27:20 GMT
Lines: 7

Jasen Mabus
RPI student

我正在寻找任何CAD格式（.dxf, .cad, .iges, .cgm等）或图片格式（.gif, .jpg, .ras等）的人脑，以用于动画演示。如果有人有或知道位置，请通过电子邮件回复mabusj@rpi.edu。

提前谢谢你，
Jasen Mabus  

处理 alt.atheism，找到 1000 个文件  
处理 comp.graphics，找到 1000 个文件  
处理 comp.os.ms-windows.misc，找到 1000 个文件  
处理 comp.sys.ibm.pc.hardware，找到 1000 个文件  
处理 comp.sys.mac.hardware，找到 1000 个文件  
处理 comp.windows.x，找到 1000 个文件  
处理 misc.forsale，找到 1000 个文件  
处理 rec.autos，找到 1000 个文件  
处理 rec.motorcycles，找到 1000 个文件  
处理 rec.sport.baseball，找到 1000 个文件  
处理 rec.sport.hockey，找到 1000 个文件  
处理 sci.crypt，找到 1000 个文件  
处理 sci.electronics，找到 1000 个文件  
处理 sci.med，找到 1000 个文件  
处理 sci.space，找到 1000 个文件  
处理 soc.religion.christian，找到 997 个文件  
处理 talk.politics.guns，找到 1000 个文件  
处理 talk.politics.mideast，找到 1000 个文件  
处理 talk.politics.misc，找到 1000 个文件  
处理 talk.religion.misc，找到 1000 个文件  
类别: ['alt.atheism', 'comp.graphics', 'comp.os.ms-windows.misc', 'comp.sys.ibm.pc.hardware', 'comp.sys.mac.hardware', 'comp.windows.x', 'misc.forsale', 'rec.autos', 'rec.motorcycles', 'rec.sport.baseball', 'rec.sport.hockey', 'sci.crypt', 'sci.electronics', 'sci.med', 'sci.space', 'soc.religion.christian', 'talk.politics.guns', 'talk.politics.mideast', 'talk.politics.misc', 'talk.religion.misc']  
样本数量: 19997
</div>
实际上有一个类别没有预期的文件数，但差异足够小，问题仍然是一个平衡的分类问题。

---
## 打乱数据并分割为训练集和验证集


```python
# 打乱数据
seed = 1337
rng = np.random.RandomState(seed)
rng.shuffle(samples)
rng = np.random.RandomState(seed)
rng.shuffle(labels)

# 提取训练和验证集的分割
validation_split = 0.2
num_validation_samples = int(validation_split * len(samples))
train_samples = samples[:-num_validation_samples]
val_samples = samples[-num_validation_samples:]
train_labels = labels[:-num_validation_samples]
val_labels = labels[-num_validation_samples:]

vectorizer = layers.TextVectorization(max_tokens=20000, output_sequence_length=200)
text_ds = tf_data.Dataset.from_tensor_slices(train_samples).batch(128)
vectorizer.adapt(text_ds)

vectorizer.get_vocabulary()[:5]

output = vectorizer([["the cat sat on the mat"]])
output.numpy()[0, :6]

voc = vectorizer.get_vocabulary()
word_index = dict(zip(voc, range(len(voc))))

test = ["the", "cat", "sat", "on", "the", "mat"]
[word_index[w] for w in test]

!wget https://downloads.cs.stanford.edu/nlp/data/glove.6B.zip
!unzip -q glove.6B.zip

path_to_glove_file = "glove.6B.100d.txt"

embeddings_index = {}
with open(path_to_glove_file) as f:
    for line in f:
        word, coefs = line.split(maxsplit=1)
        coefs = np.fromstring(coefs, "f", sep=" ")
        embeddings_index[word] = coefs

print("找到 %s 个词向量。" % len(embeddings_index))

num_tokens = len(voc) + 2
embedding_dim = 100
hits = 0
misses = 0

# 准备嵌入矩阵
embedding_matrix = np.zeros((num_tokens, embedding_dim))
for word, i in word_index.items():
    embedding_vector = embeddings_index.get(word)
    if embedding_vector is not None:
        # 嵌入索引中未找到的单词将是全零的。
        # 这包括“填充”和“OOV”的表示
        embedding_matrix[i] = embedding_vector
        hits += 1
    else:
        misses += 1
print("转换了 %d 个单词 (%d 次未找到)" % (hits, misses))

from keras.layers import Embedding

embedding_layer = Embedding(
    num_tokens,
    embedding_dim,
    trainable=False,
)
embedding_layer.build((1,))
embedding_layer.set_weights([embedding_matrix])

int_sequences_input = keras.Input(shape=(None,), dtype="int32")
embedded_sequences = embedding_layer(int_sequences_input)
x = layers.Conv1D(128, 5, activation="relu")(embedded_sequences)
x = layers.MaxPooling1D(5)(x)
x = layers.Conv1D(128, 5, activation="relu")(x)
x = layers.MaxPooling1D(5)(x)
x = layers.Conv1D(128, 5, activation="relu")(x)
x = layers.GlobalMaxPooling1D()(x)
x = layers.Dense(128, activation="relu")(x)
x = layers.Dropout(0.5)(x)
preds = layers.Dense(len(class_names), activation="softmax")(x)
model = keras.Model(int_sequences_input, preds)
model.summary()

x_train = vectorizer(np.array([[s] for s in train_samples])).numpy()
x_val = vectorizer(np.array([[s] for s in val_samples])).numpy()

y_train = np.array(train_labels)
y_val = np.array(val_labels)

model.compile(
    loss="sparse_categorical_crossentropy", optimizer="rmsprop", metrics=["acc"]
)
model.fit(x_train, y_train, batch_size=128, epochs=20, validation_data=(x_val, y_val))

string_input = keras.Input(shape=(1,), dtype="string")
x = vectorizer(string_input)
preds = model(x)
end_to_end_model = keras.Model(string_input, preds)

probabilities = end_to_end_model(
    keras.ops.convert_to_tensor(
        [["this message is about computer graphics and 3D modeling"]]
    )
)

print(class_names[np.argmax(probabilities[0])])